Accuracy of computer-aided chest X-ray screening in the Kenya National Tuberculosis Prevalence Survey

Abstract

BackgroundCommunity-based screening for tuberculosis (TB) could improve detection but is resource intensive. We set out to evaluate the accuracy of computer-aided TB screening using digital chest X-ray (CXR) to determine if this approach met target product profiles (TPP) for community-based screening.MethodsCXR images from participants in the 2016 Kenya National TB Prevalence Survey were evaluated using CAD4TBv6 (Delft Imaging), giving a probabilistic score for pulmonary TB ranging from 0 (low probability) to 99 (high probability). We constructed a Bayesian latent class model to estimate the accuracy of CAD4TBv6 screening compared to bacteriologically-confirmed TB across CAD4TBv6 threshold cut-offs, incorporating data on Clinical Officer CXR interpretation, participant demographics (age, sex, TB symptoms, previous TB history), and sputum results. We compared model-estimated sensitivity and specificity of CAD4TBv6 to optimum and minimum TPPs.ResultsOf 63,050 prevalence survey participants, 61,848 (98%) had analysable CXR images, and 8,966 (14.5%) underwent sputum bacteriological testing; 298 had bacteriologically-confirmed pulmonary TB. Median CAD4TBv6 scores for participants with bacteriologically-confirmed TB were significantly higher (72, IQR: 58-82.75) compared to participants with bacteriologically-negative sputum results (49, IQR: 44-57, p<0.0001). CAD4TBv6 met the optimum TPP; with the threshold set to achieve a mean sensitivity of 95% (optimum TPP), specificity was 83.3%, (95% credible interval [CrI]: 83.0%—83.7%, CAD4TBv6 threshold: 55). There was considerable variation in accuracy by participant characteristics, with older individuals and those with previous TB having lowest specificity.ConclusionsCAD4TBv6 met the optimal TPP for TB community screening. To optimise screening accuracy and efficiency of confirmatory sputum testing, we recommend that an adaptive approach to threshold setting is adopted based on participant characteristics.Take home messageCAD4TBv6 met the optimal WHO target product profile for a community TB screening tool. Specificity was lower in adults with previous TB and those aged 41 years or older; an adaptive approach to setting CAD thresholds will likely be required to optimize use.

The Performance of Computer-Aided Detection Digital Chest X-ray Reading Technologies for Triage of Active Tuberculosis Among Persons With a History of Previous Tuberculosis

“…This likely reflects differences in the demographic and clinical characteristics of participants [ 25 ]. Notably, we observed a significant reduction in the specificity of CAD among patients with previous TB when using a fixed abnormality threshold, which has also been reported in other studies [ 6 , 15 , 24 ].…”

“…This likely, in large part, reflects the tradeoffs between the sensitivity and specificity at a fixed abnormality threshold as the AUROCs for these subgroups were similar. However, prior studies from Bangladesh and Kenya found lower accuracy of CAD for TB among women compared with men [ 4 , 15 ]; when we applied an optimal cutoff, we found that qXR but not CAD4TB had lower specificity among women compared with men. Reduced CAD accuracy among women could, in part, be due to breast attenuation in some women, which reduces visualization of pertinent anatomy in the lower lung zones, especially if post-TB scarring is also present.…”

“…Our study adds to and extends the scarce existing evidence base on how CAD system performance as a triage tool for TB is affected by an individual’s previous TB status [ 4 , 6 , 15 , 24 ]. Similar to a study from Bangladesh that evaluated 5 AI algorithms, our study was conducted in a high-TB-burden setting, Xpert MTB/RIF was used as the reference standard, and we observed a decline in discriminatory power for CAD4TB among patients with previous TB [ 4 ].…”

Community-based active-case finding for tuberculosis: navigating a complex minefield

“…With CAD software, operators can select a "TB abnormality" threshold, above which sputum testing would be recommended. However, individual-level characteristics appear to have a large effect on CAD specificity, with older age, previous TB history, and possibly HIV infection increasing false-positive screens [68,69]. Setting adaptive thresholds based on individual characteristics could dramatically decrease the number of confirmatory sputum tests required, without impacting sensitivity; prospective studies and modelling are required.…”

Computer-aided diagnostic accuracy of pulmonary tuberculosis on chest radiography among lower respiratory tract symptoms patients

“…The acquisition of sociodemographic and clinical information including signs and symptoms of tuberculosis and a history of respiratory diseases was facilitated through the utilization of a meticulously crafted and rigorously validated questionnaire. The questionnaire's design was rooted in a thorough review of pertinent literature and established health assessment frameworks (21)(22)(23)(24)(25). This instrument was developed through a systematic process that involved collaboration with domain experts, iterative refinement, and comprehensive pilot testing.…”