Handling label noise through model confidence and uncertainty: application to chest radiograph classification

Çallı, Erdi; Sogancioglu, Ecem; Scholten, Ernst Th.; Ginneken, Bram van

doi:10.1117/12.2514290

Cited by 12 publications

(11 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, a template matching-based method as proposed in this work may be insufficient to effectively remove all the undesired samples. A more robust preprocessing technique, such as that proposed in [47], should be applied to reject almost all out-of-distribution samples.…”

Section: Discussionmentioning

confidence: 99%

Interpreting chest X-rays via CNNs that exploit disease dependencies and uncertainty labels

Pham¹,

Le²,

Tran³

et al. 2019

Preprint

View full text Add to dashboard Cite

Chest radiography is one of the most common types of diagnostic radiology exams, which is critical for screening and diagnosis of many different thoracic diseases. Specialized algorithms have been developed to detect several specific pathologies such as lung nodule or lung cancer. However, accurately detecting the presence of multiple diseases from chest X-rays (CXRs) is still a challenging task. This paper presents a supervised multi-label classification framework based on deep convolutional neural networks (CNNs) for predicting the risk of 14 common thoracic diseases. We tackle this problem by training state-of-the-art CNNs that exploit dependencies among abnormality labels. We also propose to use the label smoothing technique for a better handling of uncertain samples, which occupy a significant portion of almost every CXR dataset. Our model is trained on over 200,000 CXRs of the recently released CheXpert dataset and achieves a mean area under the curve (AUC) of 0.940 in predicting 5 selected pathologies from the validation set. This is the highest AUC score yet reported to date. The proposed method is also evaluated on the independent test set of the CheXpert competition, which is composed of 500 CXR studies annotated by a panel of 5 experienced radiologists. The performance is on average better than 2.6 out of 3 other individual radiologists with a mean AUC of 0.930, which ranks first on the CheXpert leaderboard at the time of writing this paper.

show abstract

Section: Discussionmentioning

confidence: 99%

Interpreting chest X-rays via CNNs that exploit disease dependencies and uncertainty labels

Pham¹,

Le²,

Tran³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Many studies have been done to address label noise issues in different domains such as medical imaging [31,32,33,34,35,36]. However, label noise is still an open question in computer-aided diagnosis systems.…”

Section: Discussionmentioning

confidence: 99%

Addressing Inaccurate Nosology in Mental Health: A Multi Label Data Cleansing Approach for Detecting Label Noise from Structural Magnetic Resonance Imaging Data in Mood and Psychosis Disorders

Rokham

Pearlson

Abrol

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Mental health diagnostic approaches are seeking to identify biological markers to work alongside advanced machine learning approaches. It is difficult to identify a biological marker of disease when the traditional diagnostic labels themselves are not necessarily valid. Methods: We worked with T1 structural magnetic resonance imaging data collected from individuals with mood and psychosis disorders from over 1400 individuals comprising healthy controls, psychosis patients and their unaffected first-degree relatives including 176 bipolar probands, 134 schizoaffective probands, 240 schizophrenia proband, 581 patients relatives and 362 controls. We assumed there might be noise in the diagnostic labeling process. We detected label noise by classifying the data multiple times using a support vector machine classifier, and then we flagged those individuals in which all classifiers unanimously mislabeled those subjects. Next, we assigned a new diagnostic label to these individuals, based on the biological data (MRI), using iterative data cleansing approach. Results: Simulation results showed our method was highly accurate in identifying label noise. Both diagnostic and Biotype categories showed about 65% and 63% respectively of noisy labels with the largest amount of relabeling occurring between the healthy control and bipolar and schizophrenia disorder individuals as well as in the unaffected close relatives. The extraction of imaging features highlighted regional brain changes associated with each group. Conclusions: This approach represents an initial step towards developing strategies that need not assume existing mental health diagnostic categories are always valid, but rather allows us to leverage this information while also acknowledging that there are misassignments.

show abstract

“…All images underwent per sample mean-standard deviation normalization. Data augmentation was applied to the training samples by means of inception-like preprocessing [6], [36]. This consists of applying a random rotation up to 7 degrees, random resizing with a scale in the range [0.7, 1], and random cropping a 4:3 or 3:4 part of the chest X-ray.…”

Section: A Classification-based Methodsmentioning

confidence: 99%

Cardiomegaly Detection on Chest Radiographs: Segmentation Versus Classification

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this study, we investigate the detection of cardiomegaly on frontal chest radiographs through two alternative deep-learning approaches -via anatomical segmentation and via image-level classification. We used the publicly available ChestX-ray14 dataset, and obtained heart and lung segmentation annotations for 778 chest radiographs for the development of the segmentation-based approach. The classification-based method was trained with 65k standard chest radiographs with image-level labels. For both approaches, the best models were found through hyperparameter searches where architectural, learning, and regularization related parameters were optimized systematically. The resulting models were tested on a set of 367 held-out images for which cardiomegaly annotations were hand-labeled by two independent expert radiologists. Sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC) were calculated. The performance of the segmentation-based system with an AUC of 0.977 is significantly better for classifying cardiomegaly than the classificationbased model which achieved an AUC of 0.941. Only the segmentation-based model achieved comparable performance to an independent expert reader (AUC of 0.978). We conclude that the segmentation-based model requires 100 times fewer annotated chest radiographs to achieve a substantially better performance, while also producing more interpretable results.INDEX TERMS Deep learning, chest radiographs, anatomy segmentations, cardiomegaly.

show abstract

Handling label noise through model confidence and uncertainty: application to chest radiograph classification

Cited by 12 publications

References 1 publication

Interpreting chest X-rays via CNNs that exploit disease dependencies and uncertainty labels

Interpreting chest X-rays via CNNs that exploit disease dependencies and uncertainty labels

Addressing Inaccurate Nosology in Mental Health: A Multi Label Data Cleansing Approach for Detecting Label Noise from Structural Magnetic Resonance Imaging Data in Mood and Psychosis Disorders

Cardiomegaly Detection on Chest Radiographs: Segmentation Versus Classification

Contact Info

Product

Resources

About