Performance of the TB-LAMP diagnostic assay in reference laboratories: Results from a multicentre study

Highlights • TB-LAMP sensitivity was 75.6% in samples from culture-confirmed TB cases.• TB-LAMP detected 97.9% of smear-positive TB and 46.6% of smear-negative TB.• TB-LAMP specificity was 98.7% in samples from culture-negative participants.• The performance of TB-LAMP was similar to that of Xpert MTB/RIF in this study.


Introduction
In 2016, an estimated 10.4 million people worldwide developed tuberculosis (TB) and 1.3 million HIV-negative people died from the disease (World Health Organization, 2017). The lack of rapid and accurate diagnostic tools contributed to the estimated 4.1 million cases of TB that went unreported (TDR, FIND SA, 2006). Smear microscopy, the first line in TB detection, has a low and variable sensitivity (30-70%) (Steingart et al., 2006;Aber et al., 1980;Urbanczik, 1985). Conventional culture on solid media has a sensitivity of 80-90%, but requires 2-8 weeks for a result (Lee et al., 2003;Somoskövi et al., 2000). Although conventional liquid culture (Mycobacteria Growth Indicator Tube (MGIT); Becton Dickinson Microbiology Systems) typically has sensitivity greater than 95%, the time to result is still up to 4 weeks. This results in individuals being lost to follow-up and treatment delays (Stall et al., 2011).
The use of molecular methods such as nucleic acid amplification tests (NAAT) may promise faster results with high sensitivity and specificity. The Xpert MTB/RIF assay (Cepheid, Sunnyvale, CA, USA) is an automated molecular method based on PCR. Initial demonstration studies found the sensitivity to be 77% in those with smear-negative, culture-positive TB and specificity to be >98% (Boehme et al., 2010;Boehme et al., 2011). A Cochrane review in 2014 found sensitivity in those with smear-negative, culturepositive TB to be 67% and specificity to be 99% (Steingart et al., 2014). However, Xpert MTB/RIF requires a continuous supply of electricity, hefty investment in equipment, and long-term maintenance plans.
TB-LAMP is a new, commercially available, manual molecular TB detection method based on the novel loop-mediated isothermal amplification platform (LAMP), manufactured by Eiken Chemical Co. in Japan. LAMP is an attractive diagnostics platform because it takes less than 2 h to perform, requires minimal instrumentation, and generates a fluorescent result that can be detected with the naked eye (Notomi et al., 2000). Furthermore, it has shown potential as a cross-disease platform, with assays developed for malaria, African trypanosomiasis, severe acute respiratory syndrome, and influenza (Kuboki et al., 2003;Poon et al., 2005aPoon et al., ,b, 2006. The TB-LAMP assay requires only minimal instrumentation in the form of a heating block. Additionally, it has the potential for higher throughput as it can test up to 14 samples per test run (Boehme et al., 2011;Vassall et al., 2011). A full description of the TB-LAMP procedure can be found in the recent World Health Organization (WHO) policy guidance (WHO Policy Guidance, 2016).
This study aimed to determine the accuracy of TB-LAMP in a single raw sputum sample in comparison to conventional methods and the Xpert MTB/RIF assay when performed in quality-assured TB reference laboratories.

Study population and methods
TB reference laboratories were selected in four urban centres: Cape Town in South Africa, Lima in Peru, Ho Chi Minh City in Vietnam, and Rio de Janeiro in Brazil. The reference laboratories selected were enrolled in national or international quality assurance programmes and had each undergone a laboratory assessment by FIND prior to selection. The study was approved by the institutional review boards in all countries. Informed consent was obtained from all participants. Additionally, while participants may have received better than standard diagnostic care through the use of additional cultures, clinicians were blinded to the results of TB-LAMP so as not to impact patient care.
Adults (!18 years) with symptoms suggestive of pulmonary TB, as defined by national TB programmes, were enrolled consecutively if they were able to provide two sputum samples of at least 1.5 ml and had not received TB treatment in the preceding 60 days.
Each of the two sputum samples obtained had 60 ml removed for TB-LAMP and approximately 10 ml for direct Ziehl-Neelsen (ZN) and/or light-emitting diode (LED) fluorescence microscopy (FM) smear ( Figure 1). The TB-LAMP technician was blinded to the smear results and vice versa. The remaining sputum was then processed with n-acetyl-L-cysteine-sodium hydroxide (NALCÀ ÀNaOH) and used for solid culture (Löwenstein-Jensen medium) and MGIT liquid culture (WHO, 1998;Siddiqi and Ruesch-Gerdes, 2006). The first positive culture from the two performed per sample (two samples per individual) underwent confirmation of Mycobacterium tuberculosis (MTB) complex by MPT64 antigen detection (Capilia TB; Tauns Laboratories) (Hillemann et al., 2005).
A scanty positive culture was defined as a positive MGIT at >28 days from inoculation or a Löwenstein-Jensen with <20 colonies. A culture-positive TB case was diagnosed if a participant had either two scanty or any non-scanty positive MTB culture result, consistent with previous analysis of molecular TB diagnostics (Boehme et al., 2010(Boehme et al., , 2011. Non-tuberculous mycobacteria (NTM) or mixed cases were analyzed separately.
Samples from culture-positive TB cases were further classified as smear-positive or smear-negative based on a single smear test performed on that sputum. A smear-positive TB sample required one or more acid-fast bacilli per 100 fields for ZN or per 400 fields for FM (scanty grade or higher) following WHO recommendations (World Health Organization, 2007). A smear-negative TB sample had no acid-fast bacilli detected by ZN or FM whatsoever, but was found through culture testing to be a culture-positive TB case. Samples from participants with all cultures negative and any positive smear were analyzed separately.
Any participant with all smear results negative and all four cultures negative (or up to two cultures contaminated and the remaining negative), was considered bacteriologically negative for TB. Bacteriologically negative participants found to be TB-LAMPpositive on either sputum specimen were sought for clinical and laboratory follow-up after 8 weeks. Bacteriologically negative participants found to be TB-LAMP-negative were also followed up wherever programmatically possible.
A non-TB case was defined as a participant who was bacteriologically negative and for whom no TB treatment was prescribed (based on adjunct diagnostics such as chest X-ray and/ or symptoms) at enrolment or during follow-up after 8 weeks. Bacteriologically negative participants who were treated empirically at enrolment or follow-up were considered to have clinically diagnosed TB. Such decisions were at the discretion of each physician and were subject to local variation.
Those initially bacteriologically negative with any positive culture at follow-up were reclassified as culture-positive TB. Those with no laboratory confirmation but either unimproved chest Xray (i.e., physician compared a follow-up chest X-ray to the initial chest X-ray and found it to be 'same abnormal' or 'worse') or nonremitting symptoms (i.e., physician described symptoms as 'same' or 'worse') at follow-up were reclassified as possible TB and analyzed separately.
If the TB-LAMP technician could not determine from the fluorescent read-out whether TB-LAMP was positive or negative, a second reader was asked to make the determination. If the second read was also indeterminate, a second TB-LAMP test was repeated on the same sputum sample wherever possible. Each run of up to 14 TB-LAMP tests at one time included a negative and a positive control. If the negative control is positive, this indicates potential DNA contamination and all tests in that run must be repeated after decontamination procedures. If the positive control is negative, this indicates probable reagent degradation and all tests in that run must be repeated with new reagents. The overall indeterminate rate indicates the lack of a clear result and the need for a repetition of the test from any of these causes.
The performance of TB-LAMP on a single raw sputum sample was evaluated against the reference standard of two direct ZN smears, two direct FM smears, and four cultures per patient. Because a TB-LAMP test was performed on each of the two samples submitted per participant, correlations in the results were accounted for by using bootstrapping with sampling by cluster, where each set of two samples was the cluster, to obtain exact standard errors to calculate correct 95% confidence intervals (95% CI) for a single raw sample.
The performance of the Xpert MTB/RIF assay was evaluated as a molecular test comparison to TB-LAMP. In Peru and South Africa, where the assay was already available and in use, the Xpert MTB/ RIF assay was performed on freshly concentrated sputum, whereas in Brazil and Vietnam, the Xpert assay was performed on sputum that had been frozen for 2-6 months. The Xpert MTB/RIF assay was performed after NALCÀ ÀNaOH concentration from the first sputum sample collected, according to the manufacturer's instructions ( Figure 1). In Peru, the assay was performed from a third sputum sample if provided and from the first sputum sample if not. This third sample underwent only direct ZN and FM smears before concentration with NALCÀ ÀNaOH then Xpert MTB/RIF testing. In Brazil and Vietnam, after NALCÀ ÀNaOH concentration, >0.5 ml of the remainder of the first sample collected was frozen at À70 C and tested by Xpert MTB/RIF after 2-6 months.
Any sample reported as positive for MTB by the automated output from the Xpert MTB/RIF was considered positive, while samples reported as MTB-negative were considered negative. Any error, invalid, or no result value was considered indeterminate and ineligible for sensitivity/specificity analysis. 95% CI were obtained using the binomial distribution. Blinding was deemed unnecessary given the automated nature of the test.

Results
Between January and December 2012, 1036 eligible participants were enrolled across the four sites (Table 1). Results from 25 participants were excluded from analysis for the following reasons: two for having more than two contaminated cultures; five were culture-positive but missing results of MPT64; 13 had a single positive culture after >28 days or <20 colonies; two were missing smear results; three were found to be culture-positive and NTM but only at follow-up.
Across all sites, there were 375 culture-positive TB cases, 477 non-TB participants, 43 with clinically diagnosed TB, and 69 with possible TB. These were the 964 participants and 1928 sputum samples in the primary analysis ( Figure 2). There were 38 participants with NTM (29/38) or mixed MTB-NTM infections (9/38) who were analyzed separately (Table 2). Also analyzed separately were nine FM smear-positive, culture-negative individuals.
The follow-up rate among individuals with a positive TB-LAMP and all negative smear/culture results was 75% (15/20), while the follow-up rate among participants with negative results on all tests was 67% (381/572) ( Table 1). The overall follow-up rate for those with negative smear/culture results in South Africa was 93% (169/ 182) and in Peru was 99% (153/154), while in Vietnam it was only 62% (63/102). In Brazil, 79% (11/14) of participants with a positive TB-LAMP and negative smear/culture were followed up, but none of those who were negative on all tests were followed up due to programmatic restrictions.
The overall TB-LAMP indeterminate rate was 0.3%. Two out of 2081 TB-LAMP tests performed could not be interpreted after two readings and four had a positive result from the negative control, indicating likely DNA contamination for that run of the assay. All had clear positive or negative results after repetition of the TB-LAMP assay on the same sputum.
The performance of TB-LAMP was compared to that of Xpert MTB/RIF (Table 4). In this study, overall sensitivity in smear-

Discussion
This study was successful in demonstrating the performance of the TB-LAMP assay. TB-LAMP can reliably detect nearly all smearpositive TB cases and roughly half of smear-negative TB cases from a single raw sputum sample. In this study in TB reference laboratories, no significant difference in sensitivity and specificity was found for TB-LAMP compared to Xpert MTB/RIF using four direct smears and four cultures as the reference standard.
Brazil had the lowest specificity rates for TB-LAMP and Xpert MTB/RIF, as well as the least follow-up information. Given programmatic capabilities, follow-up was only asked of participants positive on TB-LAMP and with bacteriologically negative results (79% follow-up rate); this resulted in significant improvement in the TB-LAMP specificity (94.7% excluding follow-up and 97.2% including follow-up). Directed follow-up was not performed for participants with Xpert MTB/RIF-positive and bacteriologically negative results; furthermore, the use of an imperfect standard will have a more deleterious effect on an assay with higher specificity. Although it could not be confirmed, it is possible that specimen decontamination before culture in Brazil was performed using 2% NaOH (where 1-1.5% is recommended), which would have destroyed weakly growing mycobacteria, thereby lowering culture sensitivity (Kent and Kubica, 1985;Global Laboratory Initiative, 2014). This hypothesis is further supported by the high number of smear-positive, culture-negative results found in Brazil. Finally, the laboratory in Brazil had lower sensitivity using FM (45.7%) than the more standard ZN (69.4%)the opposite of what would be expected based on a previous meta-analysis of the two methods (Steingart et al., 2006).
Excluding Brazil, specificity would be 99.4% for TB-LAMP across the other three countries and 99.7% for Xpert MTB/RIF. This specificity for Xpert MTB/RIF agrees with data reported in the literature (Steingart et al., 2014).
Peru had an unusually high TB-LAMP sensitivity in ZN smearnegatives (76.9%), while Vietnam had unusually low TB-LAMP sensitivity in FM smear-negatives (39.4%). These variations may reflect the variability of smear performance from one laboratory to another. The same variability from site to site is seen for Xpert MTB-RIF as well, supporting this conclusion.
In conclusion, while this study was limited in that complete follow-up and standardization of the protocol was not possible, the findings contribute to the growing literature on the performance of molecular assays in the intended populations. Given the finding in this study that laboratory performance of TB-LAMP approaches that of Xpert MTB/RIF, further studies are recommended in settings of intended use to evaluate these expected benefits.
The use of a manual technique may give TB-LAMP the potential to enter the market at a lower cost than its automated counterpart (Vassall et al., 2011). It is hoped that the limited infrastructure needed to reliably power a heating block for 40 min will make it possible to run this assay in decentralized settings. Furthermore, the higher throughput of this assay, with up to 14 tests per batched 2-h run, may accommodate laboratories with moderate to high workloads at any level of the health system.

Conflict of interest
Clinical evaluation and assay development was supported by FIND. The authors have no other conflicts of interest to declare.