Fairness and Explanation in AI-Informed Decision Making

Angerschmid, Alessa; Zhou, Jianlong; Theuermann, Kevin; Chen, Fang; Holzinger, Andreas

doi:10.3390/make4020026

Cited by 76 publications

(27 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both good reproducibility and clear reporting are key points to facilitate critical assessment of the model before its implementation into routine practices. This effort is pivotal in addressing ethical concerns associated with data-driven prediction tools and in guaranteeing the safety and impartiality of the prediction [ 88 ]. Ensuring the ethical aspects of integrating a data-driven model into routine clinical practice is becoming a great challenge.…”

Section: Discussionmentioning

confidence: 99%

Machine Learning–Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review

Budiarto,

Tsang,

Wilson

et al. 2023

JMIR AI

View full text Add to dashboard Cite

Background An early warning tool to predict attacks could enhance asthma management and reduce the likelihood of serious consequences. Electronic health records (EHRs) providing access to historical data about patients with asthma coupled with machine learning (ML) provide an opportunity to develop such a tool. Several studies have developed ML-based tools to predict asthma attacks. Objective This study aims to critically evaluate ML-based models derived using EHRs for the prediction of asthma attacks. Methods We systematically searched PubMed and Scopus (the search period was between January 1, 2012, and January 31, 2023) for papers meeting the following inclusion criteria: (1) used EHR data as the main data source, (2) used asthma attack as the outcome, and (3) compared ML-based prediction models’ performance. We excluded non-English papers and nonresearch papers, such as commentary and systematic review papers. In addition, we also excluded papers that did not provide any details about the respective ML approach and its result, including protocol papers. The selected studies were then summarized across multiple dimensions including data preprocessing methods, ML algorithms, model validation, model explainability, and model implementation. Results Overall, 17 papers were included at the end of the selection process. There was considerable heterogeneity in how asthma attacks were defined. Of the 17 studies, 8 (47%) studies used routinely collected data both from primary care and secondary care practices together. Extreme imbalanced data was a notable issue in most studies (13/17, 76%), but only 38% (5/13) of them explicitly dealt with it in their data preprocessing pipeline. The gradient boosting–based method was the best ML method in 59% (10/17) of the studies. Of the 17 studies, 14 (82%) studies used a model explanation method to identify the most important predictors. None of the studies followed the standard reporting guidelines, and none were prospectively validated. Conclusions Our review indicates that this research field is still underdeveloped, given the limited body of evidence, heterogeneity of methods, lack of external validation, and suboptimally reported models. We highlighted several technical challenges (class imbalance, external validation, model explanation, and adherence to reporting guidelines to aid reproducibility) that need to be addressed to make progress toward clinical adoption.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine Learning–Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review

Budiarto,

Tsang,

Wilson

et al. 2023

JMIR AI

View full text Add to dashboard Cite

show abstract

“…In real-world institutions such as courts or corrections facilities, legal terms and predictive relationships evolve, actors act adversarially, and numerous non-sampling errors can affect algorithmic performance. 86 Not only that, many decisions in ML pipelines involve “essentially contested” and value-laden constructs such as fairness 87 and justice. 88 Sociotechnically oriented data science recognizes that fairness is more than just a property of an algorithm: 89 AI can be used as part of a political agenda.…”

Section: Social Benefits Of Ethics Impact Statementsmentioning

confidence: 99%

Atomist or holist? A diagnosis and vision for more productive interdisciplinary AI ethics dialogue

2023

View full text Add to dashboard Cite

“…Other researchers found measuring fairness and reporting fairness metrics to users (i.e., the people actually interfacing with the algorithm) improved their trust of the artificial intelligence (AI) model [22]. Other research also analyzed the impact of fairness on trust; there is a synergistic effect of how fairness affects users' trust in AI and machine learning (ML) models [5].…”

Section: Prior Researchmentioning

confidence: 99%

Fairness–accuracy tradeoff: activation function choice in a neural network

McCarthy

Narayanan

2023

AI Ethics

View full text Add to dashboard Cite

Models can have different outcomes based on the different types of inputs; the training data used to build the model will change its output (a data-centric outcome), while the hyperparameter selection can also affect the model's output and performance (a model-centric outcome). When building classification models, data scientists generally focus on model performance as the key metric, often focusing on accuracy. However, other metrics should be considered during model development that assesses performance in other ways, like fairness. Assessing the fairness during the model development process is often overlooked but should be considered as part of a model's full assessment before deployment. This research investigates the fairness-accuracy tradeoff that occurred by changing only one hyperparameter in a neural network for binary classification to assess how this single change in the hidden layer can alter the algorithm's accuracy and fairness. Neural networks were used for this assessment because of their wide usage across domains, limitations to current bias measurement methods, and the underlying challenges in their interpretability. Findings suggest that assessing accuracy and fairness during model development provides value while mitigating potential negative effects for users while reducing organizational risk. No particular activation function was found to be fairer than another. Notable differences in the fairness and accuracy measures could help developers deploy a model with high accuracy and robust fairness. Algorithm development should include a grid search for hyperparameter optimization that includes fairness along with performance measures, like accuracy. While the actual choices for hyperparameters may depend on the business context and dataset considered, an optimal development process should use both fairness and model performance metrics.

show abstract

Fairness and Explanation in AI-Informed Decision Making

Cited by 76 publications

References 53 publications

Machine Learning–Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review

Machine Learning–Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review

Atomist or holist? A diagnosis and vision for more productive interdisciplinary AI ethics dialogue

Fairness–accuracy tradeoff: activation function choice in a neural network

Contact Info

Product

Resources

About