Diagnostic accuracy of SARS-CoV-2 saliva antigen testing in a real-life clinical setting

Background SARS-CoV-2 antigen tests with saliva facilitate examination in settings that lack trained personnel. However, little is known about the diagnostic accuracy in real-life clinical settings. Therefore, we studied the diagnostic accuracy of a saliva antigen test in diagnosing SARS-CoV-2 infection in a primary/secondary care testing facility. Methods Individuals who presented at a COVID-19 testing facility affiliated with a Swiss university hospital were prospectively recruited (n=377). Saliva specimen was obtained, and the PCL Inc. COVID19 Gold antigen test was conducted in parallel with 2 real-time polymerase chain reaction (RT-PCR) assays from a nasopharyngeal swab. Results RT-PCR results were positive in 53 individuals, corresponding to a prevalence of 14.1% (missing material in 1 individual). The PCL saliva antigen test was positive in 22 individuals (5.8%) and negative in 354 (93.9%). The sensitivity of the saliva antigen test was 30.2% (95% confidence interval 18.3, 44.3), both overall and in symptomatic individuals. The specificity was 98.1% (96.0, 99.3). Conclusions The diagnostic accuracy of a SARS-CoV-2 saliva antigen test in a primary/secondary care testing facility was remarkably lower than that reported in the manufacturer's specifications.


Background
Testing for SARS-CoV-2 is an essential component of the pandemic response. Rapid antigen tests using saliva were suggested as a quick, simple, comfortable, and non-invasive testing method. Only minimal training is required to conduct these tests, facilitating application in various primary care and even self-testing settings. Several studies suggested that saliva antigen tests might have an adequate performance in diagnosing SARS-CoV-2 infection ( Mattiuzzi et al., 2020 ). However, little is known about the diagnostic accuracy in real-life clinical settings , which might be significantly different from that reported in manufacturer data ( Jegerlehner et al., 2021, Mattiuzzi et al., 2020. Manufacturers of- * Correspondence: Michael Nagler, MD, PhD, MSc; phone + 41 31 664 0520; fax + 41 31 632 4862 E-mail address: michael.nagler@insel.ch (M. Nagler). # Contributed equally ten claim a sensitivity of approximately 95% (94.3% in case of the assay mentioned below).
In this prospective cross-sectional study, we assessed the diagnostic accuracy of a SARS-CoV-2 saliva antigen test in a real-life primary/secondary care setting.

Patients and Methods
This study was conducted in line with an established prospective cross-sectional study; all methodological details were described previously ( Jegerlehner et al., 2021 ). Consecutive individuals presenting at a COVID-19 testing facility affiliated with a Swiss university hospital between September and December 2021 were included, in a period when the Delta variant was predominant, at over 90%. The following inclusion criteria were applied: (a) suspected SARS-CoV-2 infection (including asymptomatic individuals following exposure), (b) age ≥18 years, and (c) signed informed consent. The flow of the individuals is given in Figure S1 of the supplementary material. The study protocol was approved by the  All numbers and percentages refer to the subset of patients indicated in the respective column (all individuals, RT-PCR positives, or RT-PCR negatives). Abbreviations: RT-PCR, real-time polymerase chain reaction; SD, standard deviation. a Individuals presenting at a COVID-19 testing facility because of symptoms consistent with COVID-19. b Individuals presenting because of exposure to individuals with SARS-CoV-2 infection. appropriate ethical committee (Kantonale Ethikkommission Bern #2020-02729). All participants signed informed consent. Clinical data were obtained using a detailed questionnaire ( CDC, 2020, Health, 2021, Jegerlehner et al., 2021. The time point of last oral intake (food or drink) was recorded. A specially trained nurse collected the saliva specimen in parallel with the nasopharyngeal swab collected for real-time polymerase chain reaction (RT-PCR). Sample material was processed within 15 minutes (saliva antigen test) or 12 hours (RT-PCR, stored at 4 °C), respectively. De-tails of the RT-PCR determination have been reported previously ( Brigger et al., 2021, Horn et al., 2022, Jegerlehner et al., 2021 ).
An immunochromatographic lateral-flow immunoassay was used for the detection of SARS-CoV-2 (COVID19 Gold; PCL Inc., Seoul, Rep. of Korea; www.pclchip.com ). SARS-CoV-2 antibodies are labeled with small gold particles and attached on a nitrocellulose membrane. The saliva antigen test was performed in parallel by a trained nurse. The instructions of the manufacturer were strictly followed (package leaflet); internal and external controls were applied. Participants were asked not to eat, drink, or smoke 30 minutes before sampling. After collecting saliva in the mouth, the participants spitted approximately 500 μL in the test tube filled with 500 μL of extraction buffer. The tubes were mixed and 2 drops were applied to the sample hole of the test card. The results were recorded after 10 minutes.
Statistical analyses were done using the Stata 14.2 statistical software (College Station, Tx: StataCorp LP). As measures of diagnostic accuracy, sensitivities and specificities were calculated with the help of a 4 × 4 table, considering the saliva antigen test as the index test and the RT-PCR as the reference standard ( Mallett et al., 2012 ).

Results
Overall, 377 participants were eventually included ( Figure S1). Most individuals presented with symptoms consistent with SARS-CoV-2 infection (n = 327; 86.7%). Fifty asymptomatic individuals were referred for workup upon exposure (13.3%). Detailed patient characteristics are given in Table 1  The number of false-negative test results was 37, and the number of false-positive test results was 6 (n = 16 true positives; n = 317 true negatives). The sensitivity of the saliva antigen test according to adapted cycle thresholds (CTs) of RT-PCR is given in Figure  1, panel B. The sensitivity ranged from 30.2% (CT 40) to 33.3% (CT 26). Among 37 individuals with false-negative test results, the time point of last food or drink intake was shorter than 30 minutes in 2 individuals (25 minutes, 20 minutes) (Fig. 1) .

Discussion
In a prospective cross-sectional study conducted in the reallife clinical setting of a primary/secondary care testing facility, the overall sensitivity of a saliva antigen test was 30.2%. Lower CT thresholds of the RT-PCR did not significantly change the sensitivity. This result is substantially lower than that reported in the manufacturer's specifications (sensitivity 94.3%).
Our results are consistent with previous studies that have shown low sensitivities of antigen tests in real-life clinical settings ( De Marinis et al., 2021, Igloi et al., 2021, Jegerlehner et al., 2021, Kritikos et al., 2021. However, our results contrast with other studies and manufacturers' data investigating antigen tests with more restricted study designs ( Graham et al., 2021 ). A Cochrane Review pointed to the limitations of these studies, major methodological concerns, and a high risk of bias ( Dinnes et al., 2020 ). As a limitation, our results were obtained using 1 particular antigen test in 1 particular setting. However, strikingly low sensitivities were observed in several studies assessing antigen tests in realistic settings ( De Marinis et al., 2021, Igloi et al., 2021, Jegerlehner et al., 2021, Kritikos et al., 2021.
In conclusion, the diagnostic accuracy of the PCL saliva antigen test in diagnosing SARS-CoV-2 infection in a primary/secondary care testing facility was considerably lower compared with that reported in manufacturer's data. This should be taken into consideration when setting up testing strategies.

Competing interests
MN received research support from Roche diagnostics outside of the present work. The other authors declare no conflict of interest.

Ethics approval
The study was approved by the local ethical committee (Kantonale Ethikkommission Bern # 2020-02729).

Availability of data
The database is available on request to the corresponding author.

Funding
MN is supported by a research grant of the Swiss National Science Foundation (#179334). The conduction of the work was supported by the Canton of Bern.

Authorship contributions
SJ collected data, wrote the manuscript, and contributed to study design and interpretation of results. FSR and PB collected data, contributed to study design and interpretation of the results, and revised the manuscript. PJ contributed to study design, interpretation of the results, and revised the manuscript. MN designed the study, analyzed and interpreted the data, and wrote the manuscript.

Supplementary materials
Supplementary material associated with this article can be found, in the online version, at doi: 10.1016/j.ijid.2022.03.037 .