Agreement Between Radiologists’ Interpretations of Screening Mammograms

Nishikawa, Robert M.; Comstock, Christopher; Linver, Michael N.; Newstead, Gillian M.; Sandhir, Vinay; Schmidt, Robert A.

doi:10.1007/978-3-319-41546-8_1

Cited by 4 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, analyzing mammograms is a challenging task. Previous works have shown the agreement between radiologists to be slight to moderate at best [2][3][4]. A second reading of mammograms by an additional radiologist has been proven to increase sensitivity and specificity [5,6].…”

Section: Radiologists Performance In Screening Digital Mammographymentioning

confidence: 99%

The Case of Missed Cancers: Applying AI as a Radiologist’s Safety Net

Chorev

Shoshan

Akselrod-Ballin³

et al. 2020

Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

View full text Add to dashboard Cite

We investigate the potential contribution of an AI system as a safety net application for radiologists in breast cancer screening. As a safety net, the AI alerts on cases suspected to be malignant which the radiologist did not recommend for a recall. We analyzed held-out data of 2,638 exams enriched with 90 missed cancers. In screening mammography settings, we show that a system alerting on 11 out of every 1,000 cases, could detect up to 10.7% of the radiologists' missed cancers. Thus, significantly increasing radiologist's sensitivity to 80.3%, while only slightly decreasing their specificity to 95.3%. Importantly, the safety net demonstrated a significant contribution to their performance even when radiologists utilized both mammography and ultrasound images. In those settings, it would have alerted 8.5 times per 1,000 cases, and detected 11.7% of the radiologists' missed cancers. In an analysis of the missed cancers by an expert, we found that most of the cancers detected by the AI were visible post-hoc. Finally, we performed a reader study with five radiologists over 120 exams, 10 of which were originally missed cancers. The AI safety net was able to assist 3 out of the 5 radiologists in detecting missed cancers without raising any false alerts.

show abstract

Section: Radiologists Performance In Screening Digital Mammographymentioning

confidence: 99%

The Case of Missed Cancers: Applying AI as a Radiologist’s Safety Net

Chorev

Shoshan

Akselrod-Ballin³

et al. 2020

Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

View full text Add to dashboard Cite

show abstract

“…Another challenge of health risk level estimation stems from the label quality. For instance, the agreement rate of the radiologists for malignancy is usually less than 80%, resulting in a noisy labeled dataset [38,47]. Despite the often-uncleared distinction between adjacent labels, it is more possible that a well-trained annotator will mislabel a Severe DR (3) sample to Moderate DR (2) rather than No DR (0).…”

Section: Introductionmentioning

confidence: 99%

Unimodal-Uniform Constrained Wasserstein Training for Medical Diagnosis

Liu

Han

Qiao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

The labels in medical diagnosis task are usually discrete and successively distributed. For example, the Diabetic Retinopathy Diagnosis (DR) involves five health risk levels: no DR (0), mild DR (1), moderate DR (2), severe DR (3) and proliferative DR (4). This labeling system is common for medical disease. Previous methods usually construct a multi-binary-classification task or propose some reparameter schemes in the output unit. In this paper, we target on this task from the perspective of loss function. More specifically, the Wasserstein distance is utilized as an alternative, explicitly incorporating the inter-class correlations by pre-defining its ground metric. Then, the ground metric which serves as a linear, convex or concave increasing function w.r.t. the Euclidean distance in a line is explored from an optimization perspective.Meanwhile, this paper also proposes of constructing the smoothed target labels that model the inlier and outlier noises by using a unimodal-uniform mixture distribution. Different from the one-hot setting, the smoothed label endues the computation of Wasserstein distance with more challenging features. With either one-hot or smoothed target label, this paper systematically concludes the practical closed-form solution. We evaluate our method on several medical diagnosis tasks (e.g., Diabetic Retinopathy and Ultrasound Breast dataset) and achieve state-of-the-art performance.

show abstract

“…• The distinction between adjacent labels is often unclear and annotator-dependent, leading to a situation where the same input may be marked with different (although probably adjacent) labels by different practitioners (or even by the same one). Radiologist usually agree on malignancy of less than 80% cases ( [8]), whereas agreement rate may be even lower on predicting individual BIRADS labels ( [10]).…”

Section: Introductionmentioning

confidence: 99%

Learning multiple non-mutually-exclusive tasks for improved classification of inherently ordered labels

Ratner,

Shoshan,

Kachman

2018

Preprint

View full text Add to dashboard Cite

Medical image classification involves thresholding of labels that represent malignancy risk levels. Usually, a task defines a single threshold, and when developing computer-aided diagnosis tools, a single network is trained per such threshold, e.g. as screening out healthy (very low risk) patients to leave possibly sick ones for further analysis (low threshold), or trying to find malignant cases among those marked as non-risk by the radiologist ("second reading", high threshold). We propose a way to rephrase the classification problem in a manner that yields several problems (corresponding to different thresholds) to be solved simultaneously. This allows the use of Multiple Task Learning (MTL) methods, significantly improving the performance of the original classifier, by facilitating effective extraction of information from existing data.

show abstract

Agreement Between Radiologists’ Interpretations of Screening Mammograms

Cited by 4 publications

References 1 publication

The Case of Missed Cancers: Applying AI as a Radiologist’s Safety Net

The Case of Missed Cancers: Applying AI as a Radiologist’s Safety Net

Unimodal-Uniform Constrained Wasserstein Training for Medical Diagnosis

Learning multiple non-mutually-exclusive tasks for improved classification of inherently ordered labels

Contact Info

Product

Resources

About