Prognostic Value and Reproducibility of AI-assisted Analysis of Lung Involvement in COVID-19 at Low-Dose Submillisievert Chest CT: Sample Size Implications for Clinical Trials

Abstract

To compare the prognostic value and reproducibility of visual versus AI-assisted analysis of lung involvement on submillisievert low-dose chest CT in COVID-19 patients. Materials and Methods: This was a HIPAA-compliant, institutional review board-approved retrospective study. From March 15 to June 1, 2020, 250 RT-PCR confirmed COVID-19 patients were studied with low-dose chest CT at admission. Visual and AI-assisted analysis of lung involvement was performed by using a semi-quantitative CT score and a quantitative percentage of lung involvement. Adverse outcome was defined as intensive care unit (ICU) admission or death. Cox regression analysis, Kaplan-Meier curves, and cross-validated receiver operating characteristic curve with area under the curve (AUROC) analysis was performed to compare model performance. Intraclass correlation coefficients (ICCs) and Bland-Altman analysis was used to assess intra-and interreader reproducibility. Results: Adverse outcome occurred in 39 patients (11 deaths, 28 ICU admissions). AUC values from AI-assisted analysis were significantly higher than those from visual analysis for both semiquantitative CT scores and percentages of lung involvement (all P<0.001). Intrareader and interreader agreement rates were significantly higher for AI-assisted analysis than visual analysis (all ICC 0.960 versus 0.885). AI-assisted variability for quantitative percentage of lung involvement was 17.2% (coefficient of variation) versus 34.7% for visual analysis. The sample size to detect a 5% change in lung involvement with 90% power and an  error of 0.05 was 250 patients with AI-assisted analysis and 1014 patients with visual analysis. Conclusion: AI-assisted analysis of lung involvement on submillisievert low-dose chest CT outperformed conventional visual analysis in predicting outcome in COVID-19 patients while I n p r e s s 3 reducing CT variability. Lung involvement on chest CT could be used as a reliable metric in future clinical trials.

Assessment of pulmonary arterial circulation 3 months after hospitalization for SARS-CoV-2 pneumonia: Dual-energy CT (DECT) angiographic study in 55 patients

“…The algorithm uses CT data to automatically identify and 3D-segment both the lung parenchyma and abnormal areas of ground-glass opacities and consolidation. The software outputs a percentage of total lung involvement (by both GGO and consolidation) [23] . Follow-up chest CTA was systematically completed by a volumetric, non-contrast, single-energy acquisition at expiration to detect air trapping, i.e., a potential confounder for perfusion defect interpretation.…”

Automated AI-Driven CT Quantification of Lung Disease Predicts Adverse Outcomes in Patients Hospitalized for COVID-19 Pneumonia

“…In accordance with Lassau et al [ 22 ], we found an association of clinical severity with increased neutrophils count and neutrophils-lymphocytes ratio. Few studies have combined clinical and biological data with chest CT for prognostication in COVID-19 pneumonia [ 14 , 21 , 22 , 23 ]. Like previous investigators, we demonstrate here that a model integrating AI-based CT scan information and clinical and biological parameters improves the prognosis performance of the score.…”

“…Despite two studies showing that lung disease extent in COVID-19 pneumonia assessed by visual scoring correlates with clinical disease severity [ 19 , 20 ], visual estimation of disease extent even done by experimented radiologists may be a source of variability. Artificial Intelligence (AI) software developed to help radiologists in the quantification of lung involvement in COVID-19 may overcome this limitation [ 21 ]. Some investigators developed their own AI system for accurate quantitative measurements and prognosis of COVID-19 pneumonia using CT [ 22 , 23 , 24 ].…”

A Pictorial Review of the Role of Imaging in the Detection, Management, Histopathological Correlations, and Complications of COVID-19 Pneumonia

“…However, several authors have recently proposed quantitative methods using open-source platforms to evaluate CT-SS and found that compared with the semi-quantitative visual score, the quantitative CT parameters have superior accuracy ( Figure 12 ) [ 152 , 159 ]. Deep learning algorithms are also promising in the evaluation and quantification of COVID-19 pneumonia [ 160 , 161 ].…”