Advertisement
Short Communication| Volume 118, P194-196, May 2022

Comparison of the clinical sensitivity and specificity of two commercial RNA SARS-CoV-2 assays

Open AccessPublished:February 22, 2022DOI:https://doi.org/10.1016/j.ijid.2022.02.032
Comparison of the clinical sensitivity and specificity of two commercial RNA SARS-CoV-2 assays
Previous ArticleAdverse events following COVID-19 vaccination in South Korea between February 28 and August 21, 2021: A nationwide observational study
Next ArticleBacterial co-infection at hospital admission in patients with COVID-19

      Highlights

      • •
        The NeuMoDx™ SARS-CoV-2 Assay showed similar performance to the reference method.
      • •
        The assay had a clinical specificity of 100% and sensitivity of 98.73%.
      • •
        The assay's limit of detection was 150 copies/mL, exceeding acceptability criteria.
      • •
        The NeuMoDx 96 Molecular System had a throughput of 144 samples over 8 hours.

      Abstract

      Objectives

      This study aimed to compare the performance of the NeuMoDx™ SARS-CoV-2 Assay, implemented on the NeuMoDx 96 Molecular System, with that of the ThermoFisher TaqPath™ COVID-19 CE-IVD RT-PCR Kit (reference method).

      Methods

      Overall, 450 nasopharyngeal swab samples, previously tested using the reference method, were tested by the NeuMoDx Assay, and the clinical sensitivity and specificity of the assay were analyzed.

      Results

      By retrospective statistical analysis of all valid results, the NeuMoDx Assay had a clinical specificity of 100% (95% confidence interval [CI]: 98.65–100.00) and a clinical sensitivity of 98.73% (95% CI: 95.47–99.85).

      Conclusions

      The NeuMoDx SARS-CoV-2 Assay demonstrated comparable analytical and clinical performance to the ThermoFisher TaqPath COVID-19 CE-IVD RT-PCR Kit. The NeuMoDx 96 Molecular System is well suited for automating medium-throughput routine SARS-CoV-2 testing or as an addition to high-throughput systems to allow fast-tracking for highly urgent clinical samples.

      Keywords

      Introduction

      In response to the COVID-19 pandemic, researchers have developed several diagnostic assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Reverse transcriptase polymerase chain reaction (RT-PCR) is the gold standard for diagnosis of active SARS-CoV-2 infections because of its high sensitivity and specificity (
      • Park G-S
      • Ku K
      • Baek S-H
      • Kim S-J
      • Kim SI
      • Kim B-T
      • et al.
      Development of Reverse Transcription Loop-Mediated Isothermal Amplification Assays Targeting Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2).
      J Mol Diagn. 2020; 22: 729-735https://doi.org/10.1016/j.jmoldx.2020.03.006
      ). Automation in molecular diagnostics enables scaling of testing capacity, which is critical for enabling a large number of tests (
      • Eigner U
      • Reucher S
      • Hefner N
      • Staffa-Peichl S
      • Kolb M
      • Betz U
      • et al.
      Clinical evaluation of multiplex RT-PCR assays for the detection of influenza A/B and respiratory syncytial virus using a high throughput system.
      J Virol Methods. 2019; 269: 49-54https://doi.org/10.1016/j.jviromet.2019.03.015
      ).
      However, many published validation studies for SARS-CoV-2 assays have low sample numbers, differences in processes, and lack of validation by independent third parties (
      • Vandenberg O
      • Martiny D
      • Rochas O
      • van Belkum A
      • Kozlakidis Z.
      Considerations for diagnostic COVID-19 tests.
      Nat Rev Microbiol. 2021; 19: 171-183https://doi.org/10.1038/s41579-020-00461-z
      ). In 2019, legislation brought UK RT-PCR testing under regulation by the United Kingdom Accreditation Service, against standards imposed by the Department of Health and Social Care (DHSC). The most stringent standard applied to travelers from destinations deemed high risk, on day 2 after arrival (

      Department of Health and Social Care. Guidance: Day 2 and day 8 PCR testing for international arrivals: minimum standards for providers. https://www.gov.uk/government/publications/minimum-standards-for-private-sector-providers-of-covid-19-testing/day-2-and-day-8-pcr-testing-for-international-arrivals-minimum-standards-for-providers, 2021 (Accessed: 7 January, 2022).

      ).
      The NeuMoDx™ SARS-CoV-2 Assay, as implemented on the NeuMoDx 96 Molecular System, is an automated, random-access, real-time RT-PCR test for detection of SARS-CoV-2 RNA, with a standard turnaround time of 80 minutes and a maximum throughput of 144 samples per 8 hours. This study evaluated the performance of the NeuMoDx Assay in comparison with the ThermoFisher TaqPath™ COVID-19 CE-IVD RT-PCR Kit, using the DHSC standards (

      Department of Health and Social Care. Guidance: Day 2 and day 8 PCR testing for international arrivals: minimum standards for providers. https://www.gov.uk/government/publications/minimum-standards-for-private-sector-providers-of-covid-19-testing/day-2-and-day-8-pcr-testing-for-international-arrivals-minimum-standards-for-providers, 2021 (Accessed: 7 January, 2022).

      ).

      Methods

      Overall, 450 blinded nasopharyngeal swabs, previously tested using the ThermoFisher TaqPath COVID‑19 CE‑IVD RT‑PCR Kit (A48067), were provided by the UK Biocentre (Milton Keynes, UK); 175 were positive and 275 were negative for SARS-CoV-2 RNA. Samples were stored at –70°C, then transported to the Harefield laboratory (Uxbridge, UK) for testing using the NeuMoDx Assay (NeuMoDx SARS-CoV-2 test strip 300800). The manufacturer's method (

      NeuMoDx Molecular I. NeuMoDx™ SARS-CoV-2 Assay Instructions For Use. https://www.fda.gov/media/136565/download, 2021 (Accessed: 7 January, 2022).

      ) was used without modification (Supplementary Methods; Table S2). Samples were collected and processed in April 2020 (1 week apart between sites), coinciding with a period of low SARS-CoV-2 community prevalence.

      Results

      Samples

      Of the 450 samples, 3 were excluded because of invalid results per the NeuMoDx manufacturer's specifications (Table S3); 447 were therefore included in the primary analysis (Table S4). Of these, 12 had discordant results between assays, related to high cycle threshold (Ct) values using the reference method (Table S5a and S5b).
      Low-level positives were excluded in a secondary analysis (Supplementary Methods); these results are presented here. In this secondary analysis, only 2 samples had discordant results.

      NeuMoDx Assay performance

      The clinical specificity of the NeuMoDx Assay was 100% (n/N=272/272; Table 1), meeting the DHSC threshold (Table S6). The sensitivity of the NeuMoDx Assay was 98.73% (n/N=155/157). Because the threshold for day 2 testing (99%) would equate to 155.43 of 157 samples, and only a whole number of samples can be positive, this was considered to satisfy the DHSC threshold (Table S6).
      Table 1Clinical sensitivity and specificity of the NeuMoDx SARS-CoV-2 Assay versus the reference method.
      Frequency (n/N)Percentage (%)Exact 2-sided 95% CI (%)
      Lower limitUpper limit
      Primary analysis: includes samples positive at the LoD per the reference method
      Specificity272/272100.0098.65100.00
      Sensitivity163/17593.1488.3396.41
      Secondary analysis: excludes samples positive at the LoD per the reference method
      Specificity272/272100.0098.65100.00
      Sensitivity155/15798.7395.4799.85
      CI, confidence interval; LoD, limit of detection.
      The NeuMoDx 96 Molecular System provided turnaround times of 80 minutes and a throughput of 144 samples every 8 hours in a routine diagnostic setting.

      Limit of detection

      The LoD for the NeuMoDx Assay was 150 copies/mL (Table S1), in agreement with the manufacturer's evaluation (

      NeuMoDx Molecular I. NeuMoDx™ SARS-CoV-2 Assay Instructions For Use. https://www.fda.gov/media/136565/download, 2021 (Accessed: 7 January, 2022).

      ). This exceeds the DHSC threshold (

      Department of Health and Social Care. Guidance: Day 2 and day 8 PCR testing for international arrivals: minimum standards for providers. https://www.gov.uk/government/publications/minimum-standards-for-private-sector-providers-of-covid-19-testing/day-2-and-day-8-pcr-testing-for-international-arrivals-minimum-standards-for-providers, 2021 (Accessed: 7 January, 2022).

      ).

      Discussion

      In our study, the clinical specificity and sensitivity of the NeuMoDx Assay demonstrated similar analytical and clinical performance to the reference method and met the acceptance criteria for all the DHSC standards (Table S6).
      The observed shift in Ct values between assays may be partially explained by sample transport and freeze-thaw. Unfortunately, because of the number of samples needed, it was impossible to mitigate this. Freeze-thaw can have marked effects: 1 study reported a 0.41 increase in Ct and a change in 10.2% of sample results after just 1 freeze-thaw cycle (
      • Li L
      • Li X
      • Guo Z
      • Wang Z
      • Zhang K
      • Li C
      • et al.
      Influence of Storage Conditions on SARS-CoV-2 Nucleic Acid Detection in Throat Swabs.
      J Infect Dis. 2020; 222: 203-205https://doi.org/10.1093/infdis/jiaa272
      ).
      The 2 discordant results could also be accounted for by freeze-thaw or transcription errors. It was not possible for either laboratory to repeat test the discordant samples; this represents a study limitation.
      Other factors potentially affecting results are the use of off-label collection devices (see Supplementary Methods), samples not being tested in parallel, and a modest sample size. Furthermore, this study assumed that the reference method is 100% sensitive and specific; however, samples were tested when prevalence was low, and therefore, the probability of false positives was higher.
      Importantly, the criteria for reference method positivity and low-level positivity were internally determined. Diagnostic laboratories frequently select cut-offs for assays, above which Ct values are considered negative (
      • Sule WF
      • Oluwayelu DO.
      Real-time RT-PCR for COVID-19 diagnosis: challenges and prospects.
      Pan Afr Med J. 2020; 35: 121https://doi.org/10.11604/pamj.supp.2020.35.24258
      ); in this case, Ct ≥30 was applied. Furthermore, most diagnostic laboratories will verify the assay's LoD before implementation into routine testing (
      • Burd EM.
      Validation of laboratory-developed molecular assays for infectious diseases.
      Clin Microbiol Rev. 2010; 23: 550-576https://doi.org/10.1128/cmr.00074-09
      ): here, the internally determined reference method LoD was 100 copies/mL compared with the manufacturer-stated 250 copies/mL (

      ThermoFisher Scientific. Thermo Fisher Scientific's TaqPath COVID-19 CE-IVD RT-PCR Kit Now Available for the Incoming International Travel Quarantine Protocol Testing in the United Kingdom. https://thermofisher.mediaroom.com/2021-05-27-Thermo-Fisher-Scientifics-TaqPath-COVID-19-CE-IVD-RT-PCR-Kit-Now-Available-for-the-Incoming-International-Travel-Quarantine-Protocol-Testing-in-the-United-Kingdom, 2021 (Accessed: 7 January, 2022).

      ), which may be due to lack of standardization of SARS-CoV-2 reference materials. The recent availability of an International Standard for SARS-CoV-2 RNA may help alleviate these challenges (

      World Health Organization. WHO/BS.2020.2402 Collaborative Study for the Establishment of a WHO International Standard for SARS-CoV-2 RNA. https://www.who.int/publications/m/item/WHO-BS-2020.2402, 2020 (Accessed: 7 January, 2022).

      ).
      Because routine SARS-CoV-2 testing remains critical, the NeuMoDx Assay has demonstrated good clinical sensitivity and specificity on a platform well suited for automated clinical testing in our laboratory.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Authors’ contributions

      ML contributed to data acquisition and analysis. PB contributed to study design, data acquisition, analysis, and interpretation. AO contributed to data acquisition, analysis, and interpretation. JK contributed to data analysis. RD contributed to study design and data interpretation. All authors were involved in drafting, critical revision, and final approval of the manuscript.

      Ethical approval

      Not applicable.

      Declaration of Competing Interest

      The authors report no competing interests.

      Acknowledgments

      The authors would like to thank staff at the UK Biocentre (Milton Keynes, UK) for the sourcing and provision of clinical samples and performance of reference method testing. The authors would also like to thank Catherine Lennox, QIAGEN Ltd, for support with the NeuMoDx SARS-CoV-2 Assay testing. Medical writing support for the development of this manuscript, under the direction of the authors, was provided by Bonnie Nicholson, PhD, of Ashfield MedComms, an Ashfield Health company, and funded by QIAGEN Manchester Ltd.

      Appendix. Supplementary materials

      References

        • Burd EM.
        Validation of laboratory-developed molecular assays for infectious diseases.
        Clin Microbiol Rev. 2010; 23: 550-576https://doi.org/10.1128/cmr.00074-09
        View in Article

      2. Department of Health and Social Care. Guidance: Day 2 and day 8 PCR testing for international arrivals: minimum standards for providers. https://www.gov.uk/government/publications/minimum-standards-for-private-sector-providers-of-covid-19-testing/day-2-and-day-8-pcr-testing-for-international-arrivals-minimum-standards-for-providers, 2021 (Accessed: 7 January, 2022).

        View in Article
        • Eigner U
        • Reucher S
        • Hefner N
        • Staffa-Peichl S
        • Kolb M
        • Betz U
        • et al.
        Clinical evaluation of multiplex RT-PCR assays for the detection of influenza A/B and respiratory syncytial virus using a high throughput system.
        J Virol Methods. 2019; 269: 49-54https://doi.org/10.1016/j.jviromet.2019.03.015
        View in Article
        • Li L
        • Li X
        • Guo Z
        • Wang Z
        • Zhang K
        • Li C
        • et al.
        Influence of Storage Conditions on SARS-CoV-2 Nucleic Acid Detection in Throat Swabs.
        J Infect Dis. 2020; 222: 203-205https://doi.org/10.1093/infdis/jiaa272
        View in Article

      5. NeuMoDx Molecular I. NeuMoDx™ SARS-CoV-2 Assay Instructions For Use. https://www.fda.gov/media/136565/download, 2021 (Accessed: 7 January, 2022).

        View in Article
        • Park G-S
        • Ku K
        • Baek S-H
        • Kim S-J
        • Kim SI
        • Kim B-T
        • et al.
        Development of Reverse Transcription Loop-Mediated Isothermal Amplification Assays Targeting Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2).
        J Mol Diagn. 2020; 22: 729-735https://doi.org/10.1016/j.jmoldx.2020.03.006
        View in Article
        • Sule WF
        • Oluwayelu DO.
        Real-time RT-PCR for COVID-19 diagnosis: challenges and prospects.
        Pan Afr Med J. 2020; 35: 121https://doi.org/10.11604/pamj.supp.2020.35.24258
        View in Article

      8. ThermoFisher Scientific. Thermo Fisher Scientific's TaqPath COVID-19 CE-IVD RT-PCR Kit Now Available for the Incoming International Travel Quarantine Protocol Testing in the United Kingdom. https://thermofisher.mediaroom.com/2021-05-27-Thermo-Fisher-Scientifics-TaqPath-COVID-19-CE-IVD-RT-PCR-Kit-Now-Available-for-the-Incoming-International-Travel-Quarantine-Protocol-Testing-in-the-United-Kingdom, 2021 (Accessed: 7 January, 2022).

        View in Article
        • Vandenberg O
        • Martiny D
        • Rochas O
        • van Belkum A
        • Kozlakidis Z.
        Considerations for diagnostic COVID-19 tests.
        Nat Rev Microbiol. 2021; 19: 171-183https://doi.org/10.1038/s41579-020-00461-z
        View in Article

      10. World Health Organization. WHO/BS.2020.2402 Collaborative Study for the Establishment of a WHO International Standard for SARS-CoV-2 RNA. https://www.who.int/publications/m/item/WHO-BS-2020.2402, 2020 (Accessed: 7 January, 2022).

        View in Article

      Article info

      Publication history

      Published online: February 22, 2022
      Accepted: February 15, 2022
      Received in revised form: February 14, 2022
      Received: January 7, 2022

      Identification

      DOI: https://doi.org/10.1016/j.ijid.2022.02.032

      Copyright

      © 2022 The Author(s). Published by Elsevier Ltd on behalf of International Society for Infectious Diseases.

      User license

      Creative Commons Attribution (CC BY 4.0) |
      How you can reuse Information Icon
      Creative Commons Attribution (CC BY 4.0)

      Permitted

      • Read, print & download
      • Redistribute or republish the final article
      • Text & data mine
      • Translate the article
      • Reuse portions or extracts from the article in other works
      • Sell or re-use for commercial purposes

      Elsevier's open access license policy

      ScienceDirect

      Access this article on ScienceDirect

      The content on this site is intended for healthcare professionals.


      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the Cookie Preference Center for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties.


      RELX