Comparison of the clinical sensitivity and specificity of two commercial RNA SARS-CoV-2 assays

Objectives This study aimed to compare the performance of the NeuMoDx™ SARS-CoV-2 Assay, implemented on the NeuMoDx 96 Molecular System, with that of the ThermoFisher TaqPath™ COVID-19 CE-IVD RT-PCR Kit (reference method). Methods Overall, 450 nasopharyngeal swab samples, previously tested using the reference method, were tested by the NeuMoDx Assay, and the clinical sensitivity and specificity of the assay were analyzed. Results By retrospective statistical analysis of all valid results, the NeuMoDx Assay had a clinical specificity of 100% (95% confidence interval [CI]: 98.65–100.00) and a clinical sensitivity of 98.73% (95% CI: 95.47–99.85). Conclusions The NeuMoDx SARS-CoV-2 Assay demonstrated comparable analytical and clinical performance to the ThermoFisher TaqPath COVID-19 CE-IVD RT-PCR Kit. The NeuMoDx 96 Molecular System is well suited for automating medium-throughput routine SARS-CoV-2 testing or as an addition to high-throughput systems to allow fast-tracking for highly urgent clinical samples.


Introduction
In response to the COVID-19 pandemic, researchers have developed several diagnostic assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Reverse transcriptase polymerase chain reaction (RT-PCR) is the gold standard for diagnosis of active SARS-CoV-2 infections because of its high sensitivity and specificity ( Park et al., 2020 ). Automation in molecular diagnostics enables scaling of testing capacity, which is critical for enabling a large number of tests ( Eigner et al., 2019 ).
However, many published validation studies for SARS-CoV-2 assays have low sample numbers, differences in processes, and lack of validation by independent third parties ( Vandenberg et al., 2021 ). In 2019, legislation brought UK RT-PCR testing under regulation by the United Kingdom Accreditation Service, against standards imposed by the Department of Health and Social Care (DHSC). The most stringent standard applied to travelers from destinations deemed high risk, on day 2 after arrival ( Department of Health and Social Care, 2021 ).
The NeuMoDx TM SARS-CoV-2 Assay, as implemented on the NeuMoDx 96 Molecular System, is an automated, random-access, real-time RT-PCR test for detection of SARS-CoV-2 RNA, with a standard turnaround time of 80 minutes and a maximum throughput of 144 samples per 8 hours. This study evaluated the performance of the NeuMoDx Assay in comparison with the Ther-moFisher TaqPath TM COVID-19 CE-IVD RT-PCR Kit, using the DHSC standards ( Department of Health and Social Care, 2021 ).

Samples
Of the 450 samples, 3 were excluded because of invalid results per the NeuMoDx manufacturer's specifications ( Table S3 ); 447 were therefore included in the primary analysis ( Table S4 ). Of these, 12 had discordant results between assays, related to high cycle threshold (Ct) values using the reference method ( Table S5a and S5b ).
Low-level positives were excluded in a secondary analysis ( Supplementary Methods ); these results are presented here. In this secondary analysis, only 2 samples had discordant results.

NeuMoDx Assay performance
The clinical specificity of the NeuMoDx Assay was 100% (n/N = 272/272; Table 1 ), meeting the DHSC threshold ( Table S6 ). The sensitivity of the NeuMoDx Assay was 98.73% (n/N = 155/157). Because the threshold for day 2 testing (99%) would equate to 155.43 of 157 samples, and only a whole number of samples can be positive, this was considered to satisfy the DHSC threshold ( Table S6 ).
The NeuMoDx 96 Molecular System provided turnaround times of 80 minutes and a throughput of 144 samples every 8 hours in a routine diagnostic setting.

L imit of detection
The LoD for the NeuMoDx Assay was 150 copies/mL ( Table S1 ), in agreement with the manufacturer's evaluation ( NeuMoDx Molecular, 2021 ). This exceeds the DHSC threshold ( Department of Health and Social Care, 2021 ).

Discussion
In our study, the clinical specificity and sensitivity of the Neu-MoDx Assay demonstrated similar analytical and clinical performance to the reference method and met the acceptance criteria for all the DHSC standards ( Table S6 ).
The observed shift in Ct values between assays may be partially explained by sample transport and freeze-thaw. Unfortunately, because of the number of samples needed, it was impossible to mitigate this. Freeze-thaw can have marked effects: 1 study reported a 0.41 increase in Ct and a change in 10.2% of sample results after just 1 freeze-thaw cycle ( Li et al., 2020 ).
The 2 discordant results could also be accounted for by freezethaw or transcription errors. It was not possible for either laboratory to repeat test the discordant samples; this represents a study limitation.
Other factors potentially affecting results are the use of off-label collection devices (see Supplementary Methods ), samples not being tested in parallel, and a modest sample size. Furthermore, this study assumed that the reference method is 100% sensitive and specific; however, samples were tested when prevalence was low, and therefore, the probability of false positives was higher.
Importantly, the criteria for reference method positivity and low-level positivity were internally determined. Diagnostic laboratories frequently select cut-offs for assays, above which Ct values are considered negative ( Sule and Oluwayelu, 2020 ); in this case, Ct ≥30 was applied. Furthermore, most diagnostic laboratories will verify the assay's LoD before implementation into routine testing ( Burd, 2010 ): here, the internally determined reference method LoD was 100 copies/mL compared with the manufacturerstated 250 copies/mL ( ThermoFisher Scientific, 2021 ), which may be due to lack of standardization of SARS-CoV-2 reference materials. The recent availability of an International Standard for SARS-CoV-2 RNA may help alleviate these challenges ( World Health Organization, 2020 ).
Because routine SARS-CoV-2 testing remains critical, the Neu-MoDx Assay has demonstrated good clinical sensitivity and specificity on a platform well suited for automated clinical testing in our laboratory.

Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors' contributions
ML contributed to data acquisition and analysis. PB contributed to study design, data acquisition, analysis, and interpretation. AO contributed to data acquisition, analysis, and interpretation. JK contributed to data analysis. RD contributed to study design and data interpretation. All authors were involved in drafting, critical revision, and final approval of the manuscript.

Ethical approval
Not applicable.

Declaration of Competing Interest
The authors report no competing interests.