Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa

Fehr, Jana; Konigorski, Stefan; Olivier, Stephen; Gunda, Resign; Surujdeen, Ashmika; Gareta, Dickman; Smit, Theresa; Harling, Guy; Moodley, Sashen; Pillay, Deenan; Grant, Alison D.; Siedner, Mark J.; Lippert, Christoph; Wong, Emily; Team, Vukuzazi

doi:10.1101/2020.09.04.20188045

Cited by 15 publications

(20 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CAD4TB, Lunit INSIGHT CXR, and qXR's AUROCs in this study are slightly lower than those in previous independent evaluations. 17,26 Our subanalysis showed that the performance of computer-aided detection varied with demographic and clinical factors, as well as patient source, and therefore implied that variation exists in AI performance across different contexts and geographies. In our study population, rates of bacteriological positivity and rifampicin resistance are both higher than the national average because of the high proportion of referrals and urban population in this study.…”

Section: Discussionmentioning

confidence: 90%

Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms

Qin¹,

Ahmed

Sarker

et al. 2021

The Lancet Digital Health

146

123

View full text Add to dashboard Cite

Background Artificial intelligence (AI) algorithms can be trained to recognise tuberculosis-related abnormalities on chest radiographs. Various AI algorithms are available commercially, yet there is little impartial evidence on how their performance compares with each other and with radiologists. We aimed to evaluate five commercial AI algorithms for triaging tuberculosis using a large dataset that had not previously been used to train any AI algorithms.Methods Individuals aged 15 years or older presenting or referred to three tuberculosis screening centres in Dhaka, Bangladesh, between May 15, 2014, and Oct 4, 2016, were recruited consecutively. Every participant was verbally screened for symptoms and received a digital posterior-anterior chest x-ray and an Xpert MTB/RIF (Xpert) test. All chest x-rays were read independently by a group of three registered radiologists and five commercial AI algorithms: CAD4TB (version 7), InferRead DR (version 2), Lunit INSIGHT CXR (version 4.9.0), JF CXR-1 (version 2), and qXR (version 3). We compared the performance of the AI algorithms with each other, with the radiologists, and with the WHO's Target Product Profile (TPP) of triage tests (≥90% sensitivity and ≥70% specificity). We used a new evaluation framework that simultaneously evaluates sensitivity, proportion of Xpert tests avoided, and number needed to test to inform implementers' choice of software and selection of threshold abnormality scores. Findings Chest x-rays from 23 954 individuals were included in the analysis. All five AI algorithms significantly outperformed the radiologists. The areas under the receiver operating characteristic curve were 90•81% (95% CI 90•33-91•29) for qXR, 90•34% (89•81-90•87) for CAD4TB, 88•61% (88•03-89•20) for Lunit INSIGHT CXR, 84•90% (84•27-85•54) for InferRead DR, and 84•89% (84•26-85•53) for JF CXR-1. Only qXR (74•3% specificity [95% CI 73•3-74•9]) and CAD4TB (72•9% specificity [72•3-73•5]) met the TPP at 90% sensitivity. All five AI algorithms reduced the number of Xpert tests required by 50% while maintaining a sensitivity above 90%. All AI algorithms performed worse among older age groups (>60 years) and people with a history of tuberculosis.Interpretation AI algorithms can be highly accurate and useful triage tools for tuberculosis detection in high-burden regions, and outperform human readers.Funding Government of Canada.

show abstract

Section: Discussionmentioning

confidence: 90%

Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms

Qin¹,

Ahmed

Sarker

et al. 2021

The Lancet Digital Health

146

123

View full text Add to dashboard Cite

show abstract

“…Our screening archive contained data from a variety of risk groups that are recommended for targeted TB screening given their high TB prevalence. Very few peer-reviewed studies have assessed the performance of CADs in such groups, and all studies except one (23) used earlier versions of CAD4TB (24–26). Two of these studies suggested that CAD4TB v5 or v6 could achieve a similar performance as that of a trained field radiologist when interpreting CXRs of the Zambia National TB prevalence survey (25), or a senior radiologist when interpreting images from a population-based survey in a high HIV-endemic area in rural South Africa (23).…”

Section: Discussionmentioning

confidence: 99%

Diagnostic accuracy of chest X-ray interpretation for tuberculosis by three artificial intelligence-based software in a screening use-case: an individual patient meta-analysis of global data

Kik

Gelaw

Rühwald

et al. 2022

Preprint

View full text Add to dashboard Cite

Background Chest X-ray (CXR) screening is a useful diagnostic tool to test individuals at high risk of tuberculosis (TB), yet image interpretation requires trained human readers who are in short supply in many high TB burden countries. Therefore, CXR interpretation by computer-aided detection software (CAD) may overcome some of these challenges, but evidence on its accuracy is still limited. We established a CXR library with images and metadata from individuals and risk groups that underwent TB screening in a variety of countries to assess the diagnostic accuracy of three commercial CAD solutions through an individual participant meta-analysis. Methods and findings We collected digital CXRs and demographic and clinical data from 6 source studies involving a total of 2756 participants, 1753 (64%) of whom also had microbiological test information. All CXR images were analyzed with CAD4TB v6 (Delft Imaging), Lunit Insight CXR TB algorithm v4.9.0 (Lunit Inc.), and qXR v2 (Qure.ai) and re-read by an expert radiologist who was blinded to the initial CXR reading, the CAD scores, and participant information. While the performance of CAD varied across source studies, the pooled, meta-analyzed summary receiver operating characteristic (ROC) curves of the three products against a microbiological reference standard were similar, with area under the curves (AUCs) of 76.4 (95% CI 72.1-80.3) for CAD4TB, 83.3 (95% CI 78.4-87.2) for Lunit, and 76.4 (95% CI 72.1-80.3) for qXR. None of the CAD products, or the radiologists, met the targets for a triage test of 90% sensitivity and 70% specificity. At the same sensitivity of the expert radiologist (94.0%), all CAD had slightly lower point estimates for specificity (22.4% (95% CI 16.9-29.0) for CAD4TB, 34.6% (95% CI 25.3-45.1) for qXR, and 41.0% (95% CI 30.1-53.0) for Lunit compared to 45.6% for the expert radiologist). At the same specificity of 45.6%, all CAD products had lower point estimates for sensitivity but overlapping CIs with the sensitivity estimate of the radiologist. Conclusions We showed that, overall, three commercially available CAD products had a reasonable diagnostic accuracy for microbiologically confirmed pulmonary TB and may achieve a sensitivity and specificity that approximates those of experienced radiologists. While threshold setting and cost-effectiveness modelling are needed to inform the optimal implementation of CAD products as part of screening programs, the availability of CAD will assist in scaling up active case finding for TB and hence contribute to TB elimination in these settings.

show abstract

“…Several large-scale, community-based CXR screening initiatives have been recently implemented in high TB burden countries 6 – 11 . These programs show that this case finding approach is both feasible to implement in low- and middle-income countries (LMIC) and effective at identifying people with TB, particularly those with subclinical disease who are frequently missed by TB programs or only diagnosed after long delays 12 .…”

Section: Introductionmentioning

confidence: 99%

“…The majority of the published literature on CAD software for TB screening has focused on Delft Imaging’s CAD4TB (The Netherlands), which was one of the first commercially available CAD solutions 8 , 11 , 20 – 23 . Two systematic reviews, conducted in 2016 and 2019, also primarily included studies evaluating various versions of the CAD4TB software 24 , 25 .…”

Section: Introductionmentioning

confidence: 99%

Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis

Codlin¹,

Dao²,

Vo³

et al. 2021

Sci Rep

View full text Add to dashboard Cite

There have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with different levels of experience and then processed by 12 CAD software solutions. Using Xpert MTB/RIF results as the reference standard, we compared the performance characteristics of each CAD software against both an Expert and Intermediate Reader, using cut-off thresholds which were selected to match the sensitivity of each human reader. Six CAD systems performed on par with the Expert Reader (Qure.ai, DeepTek, Delft Imaging, JF Healthcare, OXIPIT, and Lunit) and one additional software (Infervision) performed on par with the Intermediate Reader only. Qure.ai, Delft Imaging and Lunit were the only software to perform significantly better than the Intermediate Reader. The majority of these CAD software showed significantly lower performance among participants with a past history of TB. The radiography equipment used to capture the CXR image was also shown to affect performance for some CAD software. TB program implementers now have a wide selection of quality CAD software solutions to utilize in their CXR screening initiatives.

show abstract

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa

Cited by 15 publications

References 42 publications

Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms

Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms

Diagnostic accuracy of chest X-ray interpretation for tuberculosis by three artificial intelligence-based software in a screening use-case: an individual patient meta-analysis of global data

Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis

Contact Info

Product

Resources

About