Risk of Training Diagnostic Algorithms on Data with Demographic Bias

Abbasi-Sureshjani, Samaneh; Raumanns, Ralf; Michels, Britt E. J.; Schouten, Greetje; Cheplygina, Veronika

doi:10.1007/978-3-030-61166-8_20

Cited by 26 publications

(20 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, the first issue one encounters is that a large number of candidate measures exist. One can for instance evaluate fairness by comparing standard ML performance metrics across different sub-groups, such as accuracy 10,[12][13][14][15][16] , or AUC ROC (the area under the receiver operating characteristic curve) [8][9][10][14][15][16][17][18][19][20][21][22] , among others. Alternatively, one can choose to employ one of the (no less than ten) different fairnessspecific criteria formulated by the community 23 in order to audit the presence of bias in a given model 16,18 .…”

Section: What Does It Mean For An Algorithm To Be Fair?mentioning

confidence: 99%

See 1 more Smart Citation

Addressing fairness in artificial intelligence for medical imaging

2022

View full text Add to dashboard Cite

A plethora of work has shown that AI systems can systematically and unfairly be biased against certain populations in multiple scenarios. The field of medical imaging, where AI systems are beginning to be increasingly adopted, is no exception. Here we discuss the meaning of fairness in this area and comment on the potential sources of biases, as well as the strategies available to mitigate them. Finally, we analyze the current state of the field, identifying strengths and highlighting areas of vacancy, challenges and opportunities that lie ahead.

show abstract

Section: What Does It Mean For An Algorithm To Be Fair?mentioning

confidence: 99%

“…Several studies in recent years have proposed solutions to mitigate bias and develop fairer algorithms 10,11,[14][15][16][17]19,20,24 . There are three main stages at which bias mitigation strategies can be adopted 11 : before, during and after training.…”

Section: Bias Mitigation Strategiesmentioning

confidence: 99%

Addressing fairness in artificial intelligence for medical imaging

2022

View full text Add to dashboard Cite

show abstract

“…Indeed, a good overall prediction performance can be achieved despite a poor performance on a minority group. Ensuring that a predictor performs well for all subpopulations reduces sensitivity to potential shifts in demographics and is essential to ensure fairness [ 35 ]. For instance, there is a risk that machine-learning analysis of dermoscopic images under-diagnoses malignant moles on skin tones that are typically under-represented in the training set [ 56 ].…”

Section: Other Approaches To Dataset Shiftmentioning

confidence: 99%

Preventing dataset shift from breaking machine-learning biomarkers

2021

View full text Add to dashboard Cite

Machine learning brings the hope of finding new biomarkers extracted from cohorts with rich biomedical measurements. A good biomarker is one that gives reliable detection of the corresponding condition. However, biomarkers are often extracted from a cohort that differs from the target population. Such a mismatch, known as a dataset shift, can undermine the application of the biomarker to new individuals. Dataset shifts are frequent in biomedical research, e.g., because of recruitment biases. When a dataset shift occurs, standard machine-learning techniques do not suffice to extract and validate biomarkers. This article provides an overview of when and how dataset shifts break machine-learning–extracted biomarkers, as well as detection and correction strategies.

show abstract

“…[Oakden- Rayner et al, 2020, Gianfrancesco et al, 2018, Barocas et al, 2019, Abbasi-Sureshjani et al, 2020, Cirillo et al, 2020.…”

Section: Preferential Sample Selection: a Common Source Of Shiftmentioning

confidence: 99%

“…Indeed, a good overall prediction performance can be achieved despite a poor performance on a minority group. Ensuring that a predictor performs well for all subpopulations reduces sensitivity to potential shifts in demographics and is essential to ensure fairness [Abbasi-Sureshjani et al, 2020]. For instance, there is a risk that machine-learning analysis of dermoscopic images underdiagnoses malignant moles on skin tones that are typically under-represented in the training set Adamson and Smith [2018].…”

Section: Performance Heterogeneity and Fairnessmentioning

confidence: 99%

Preventing dataset shift from breaking machine-learning biomarkers

Dockès,

Varoquaux,

Poline

2021

Preprint

View full text Add to dashboard Cite

Machine learning brings the hope of finding new biomarkers extracted from cohorts with rich biomedical measurements. A good biomarker is one that gives reliable detection of the corresponding condition. However, biomarkers are often extracted from a cohort that differs from the target population. Such a mismatch, known as a dataset shift, can undermine the application of the biomarker to new individuals. Dataset shifts are frequent in biomedical research, e.g. because of recruitment biases. When a dataset shift occurs, standard machine-learning techniques do not suffice to extract and validate biomarkers. This article provides an overview of when and how dataset shifts breaks machine-learning extracted biomarkers, as well as detection and correction strategies.

show abstract

Risk of Training Diagnostic Algorithms on Data with Demographic Bias

Cited by 26 publications

References 25 publications

Addressing fairness in artificial intelligence for medical imaging

Addressing fairness in artificial intelligence for medical imaging

Preventing dataset shift from breaking machine-learning biomarkers

Preventing dataset shift from breaking machine-learning biomarkers

Contact Info

Product

Resources

About