A citizen science approach to optimising computer aided detection (CAD) in mammography

Ionescu, Georgia Valeria; Hulleman, Johan; Astley, Susan M.

doi:10.1117/12.2293668

Cited by 3 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…18 Overall sensitivity increased by 15.8% from pre-CAD to post-CAD, which is in agreement with previous work with both experts and non-experts. [4][5][6][7][8][9][10] These studies have also reported significant reductions in sensitivity for targets that CAD failed to mark. However, in this study there was no significant reduction in sensitivity for these targets.…”

Section: Discussionmentioning

confidence: 91%

“…Previous work in our group has investigated the distracting effect of false positive CAD prompts in a citizen science study. 10 For unmarked targets, the presence of false positive prompts led to a lower detection rate than that in images without false prompts. Whereas in this study, there was no significant difference between unmarked target detectability for images with and without false CAD prompts.…”

Section: Discussionmentioning

confidence: 96%

“…Readers' trust in CAD software is an important factor in the effectiveness of improving cancer detection as it has been shown that 71% of correct CAD prompts are ignored, 1 but there is also a risk of overreliance which can lead to large increases in recall rate. 3 Observer studies with expert readers 4-6 and non-expert readers [7][8][9][10] have investigated the effect of CAD prompts on observer performance and visual search, and all found improvements to overall sensitivity with CAD. However, there are disadvantages: for the targets that were not marked by CAD (either because there were no prompts on the image or prompts in an incorrect location), sensitivity was significantly lower than observer sensitivity without CAD.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Is there a safety-net effect with computer-aided detection (CAD)?

Du-Crow¹,

Warren

Astley³

et al. 2019

Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment

Self Cite

View full text Add to dashboard Cite

Computer-Aided Detection (CAD) systems are used to aid readers interpreting screening mammograms. An expert reader searches the image initially unaided, and then once again with the aid of CAD which prompts automatically detected suspicious regions. This could lead to a 'safety-net' effect, where the initial unaided search of the image is adversely affected by the fact that it is preliminary to an additional search with CAD, and may, therefore, be less thorough. To investigate the existence of such an effect, we created a visual search experiment for non-expert observers mirroring breast screening with CAD. Each observer searched 100 images for microcalcification clusters within synthetic images in both prompted and unprompted (no-CAD) conditions. Fifty-two participants were recruited for the study, 48 of whom had their eye movements tracked in real-time; four participants could not be accurately calibrated so only behavioural data was collected. In the CAD condition, before prompts were displayed, image coverage was significantly lower than coverage in the no-CAD condition (t(47)=5.48, p<0.001). Observer sensitivity was significantly greater for targets marked by CAD than the same targets in the no-CAD condition (t(51)=11.67, p<0.001). For targets not marked by CAD, there was no significant difference in observer sensitivity in the CAD condition compared to the same targets in the no-CAD condition (t(51)=0.88, p=0.382). These results suggest that the initial search may be influenced by the subsequent availability of CAD; if so, CAD efficacy studies should account for the effect when estimating benefit.

show abstract

Section: Discussionmentioning

confidence: 91%

Section: Discussionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Is there a safety-net effect with computer-aided detection (CAD)?

Du-Crow¹,

Warren

Astley³

et al. 2019

Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

Is there a safety-net effect with computer-aided detection?

Du-Crow

Astley²,

Hulleman

2019

J. Med. Imag.

View full text Add to dashboard Cite

Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification

Spicher,

Wesemeyer,

Deserno

2023

Biomedical Engineering / Biomedizinische Technik

View full text Add to dashboard Cite

Objectives Segmentation is crucial in medical imaging. Deep learning based on convolutional neural networks showed promising results. However, the absence of large-scale datasets and a high degree of inter- and intra-observer variations pose a bottleneck. Crowdsourcing might be an alternative, as many non-experts provide references. We aim to compare different types of crowdsourcing for medical image segmentation. Methods We develop a crowdsourcing platform that integrates citizen science (incentive: participating in the research), paid microtask (incentive: financial reward), and gamification (incentive: entertainment). For evaluation, we choose the use case of sclera segmentation in fundus images as a proof-of-concept and analyze the accuracy of crowdsourced masks and the generalization of learning models trained with crowdsourced masks. Results The developed platform is suited for the different types of crowdsourcing and offers an easy and intuitive way to implement crowdsourcing studies. Regarding the proof-of-concept study, citizen science, paid microtask, and gamification yield a median F-score of 82.2, 69.4, and 69.3 % compared to expert-labeled ground truth, respectively. Generating consensus masks improves the gamification masks (78.3 %). Despite the small training data (50 images), deep learning reaches median F-scores of 80.0, 73.5, and 76.5 % for citizen science, paid microtask, and gamification, respectively, indicating sufficient generalizability. Conclusions As the platform has proven useful, we aim to make it available as open-source software for other researchers.

show abstract

A citizen science approach to optimising computer aided detection (CAD) in mammography

Cited by 3 publications

References 16 publications

Is there a safety-net effect with computer-aided detection (CAD)?

Is there a safety-net effect with computer-aided detection (CAD)?

Is there a safety-net effect with computer-aided detection?

Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification

Contact Info

Product

Resources

About