The 2021 SIIM-FISABIO-RSNA Machine Learning COVID-19 Challenge: Annotation and Standard Exam Classification of COVID-19 Chest Radiographs

Lakhani, Paras; Mongan, John; Singhal, C.; Zhou, Qingtao; Andriole, Katherine P.; Auffermann, William F.; Prasanna, Prasanth; Pham, T. X.; Peterson, Michael R.; Bergquist, P. J.; Cook, Tessa S.; Ferraciolli, Suely Fazio; Corradi, Gustavo C.A.; Takahashi, MS; Workman, C. S.; Parekh, Maansi; Kamel, Serageldin; Galant, J.; Mas-Sanchez, A.; Benítez, E. C.; Sánchez-Valverde, M.; Jaques, L.; Panadero, M.; Vidal, Marta; Culiañez-Casas, M.; Angulo-Gonzalez, D.; Langer, S. G.; Iglesia-Vayá, María de la; Shih, George

doi:10.1007/s10278-022-00706-8

Cited by 14 publications

(24 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, many studies used ML/DL to prognosticate patients with COVID-19 and predict severe outcomes, like ICU admission or death [ 31 ]. Publicly available datasets and coding challenges fueled this enthusiasm by creating a way to benchmark algorithm performance [ 32 , 33 , 34 ].…”

Section: Results and Synthesismentioning

confidence: 99%

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

et al. 2022

View full text Add to dashboard Cite

Machine-learning (ML) and deep-learning (DL) algorithms are part of a group of modeling algorithms that grasp the hidden patterns in data based on a training process, enabling them to extract complex information from the input data. In the past decade, these algorithms have been increasingly used for image processing, specifically in the medical domain. Cardiothoracic imaging is one of the early adopters of ML/DL research, and the COVID-19 pandemic resulted in more research focus on the feasibility and applications of ML/DL in cardiothoracic imaging. In this scoping review, we systematically searched available peer-reviewed medical literature on cardiothoracic imaging and quantitatively extracted key data elements in order to get a big picture of how ML/DL have been used in the rapidly evolving cardiothoracic imaging field. During this report, we provide insights on different applications of ML/DL and some nuances pertaining to this specific field of research. Finally, we provide general suggestions on how researchers can make their research more than just a proof-of-concept and move toward clinical adoption.

show abstract

Section: Results and Synthesismentioning

confidence: 99%

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The models developed in this work were trained using the SIIM-FISABIO-RSNA COVID-19 Detection data set [ 17 ], which is also the main corpus used recurrently in the state-of-the-art. The data set was compiled from two public sources: BIMCV [ 30 ] and MIDRC-RICORD [ 10 ].…”

Section: Methodsmentioning

confidence: 99%

“…Every image was annotated only by a single radiologist out of 22. The experts received specific training and evaluation to agree on the criteria used to label the data set [ 17 ]. To assess the consistency of the evaluation, a 25 CXR test case was performed, obtaining a median percentage agreement of 86% between experts.…”

Section: Methodsmentioning

confidence: 99%

“… Examples of the three pre-processing schemes used. ( Left ): Raw image with equalization (taken from [ 17 ]). ( Centre ): Cropped image.…”

Section: Figurementioning

confidence: 99%

“…The challenge’s goal was twofold, including lesion detection and classification into four categories. Participants trained their models using the SIIM-FISABIO-RSNA COVID-19 Detection data set [ 17 ], which contains 6334 CXR images. The data set was manually annotated by experts with the corresponding bounding boxes for the lesions observed in the images.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Identification of Lung Opacities Due to COVID-19 from Chest X-ray Images—Focussing Attention on the Lungs

2023

View full text Add to dashboard Cite

Due to the primary affection of the respiratory system, COVID-19 leaves traces that are visible in plain chest X-ray images. This is why this imaging technique is typically used in the clinic for an initial evaluation of the patient’s degree of affection. However, individually studying every patient’s radiograph is time-consuming and requires highly skilled personnel. This is why automatic decision support systems capable of identifying those lesions due to COVID-19 are of practical interest, not only for alleviating the workload in the clinic environment but also for potentially detecting non-evident lung lesions. This article proposes an alternative approach to identify lung lesions associated with COVID-19 from plain chest X-ray images using deep learning techniques. The novelty of the method is based on an alternative pre-processing of the images that focuses attention on a certain region of interest by cropping the original image to the area of the lungs. The process simplifies training by removing irrelevant information, improving model precision, and making the decision more understandable. Using the FISABIO-RSNA COVID-19 Detection open data set, results report that the opacities due to COVID-19 can be detected with a Mean Average Precision with an IoU > 0.5 (mAP@50) of 0.59 following a semi-supervised training procedure and an ensemble of two architectures: RetinaNet and Cascade R-CNN. The results also suggest that cropping to the rectangular area occupied by the lungs improves the detection of existing lesions. A main methodological conclusion is also presented, suggesting the need to resize the available bounding boxes used to delineate the opacities. This process removes inaccuracies during the labelling procedure, leading to more accurate results. This procedure can be easily performed automatically after the cropping stage.

show abstract

Improving model fairness in image-based computer-aided diagnosis

Lin,

Li,

Yang

et al. 2023

Nat Commun

View full text Add to dashboard Cite

Deep learning has become a popular tool for computer-aided diagnosis using medical images, sometimes matching or exceeding the performance of clinicians. However, these models can also reflect and amplify human bias, potentially resulting inaccurate missed diagnoses. Despite this concern, the problem of improving model fairness in medical image classification by deep learning has yet to be fully studied. To address this issue, we propose an algorithm that leverages the marginal pairwise equal opportunity to reduce bias in medical image classification. Our evaluations across four tasks using four independent large-scale cohorts demonstrate that our proposed algorithm not only improves fairness in individual and intersectional subgroups but also maintains overall performance. Specifically, the relative change in pairwise fairness difference between our proposed model and the baseline model was reduced by over 35%, while the relative change in AUC value was typically within 1%. By reducing the bias generated by deep learning models, our proposed approach can potentially alleviate concerns about the fairness and reliability of image-based computer-aided diagnosis.

show abstract

The 2021 SIIM-FISABIO-RSNA Machine Learning COVID-19 Challenge: Annotation and Standard Exam Classification of COVID-19 Chest Radiographs

Cited by 14 publications

References 32 publications

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

Automatic Identification of Lung Opacities Due to COVID-19 from Chest X-ray Images—Focussing Attention on the Lungs

Improving model fairness in image-based computer-aided diagnosis

Contact Info

Product

Resources

About