A Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning

McCradden, Melissa D; Anderson, James A.; Stephenson, Elizabeth A.; Drysdale, Erik; Erdman, Lauren; Goldenberg, Anna; Shaul, Randi Zlotnik

doi:10.1080/15265161.2021.2013977

Cited by 57 publications

(32 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although a preliminary data screening procedure was performed to include candidate predictors that are health-related and clinically meaningful (eTable 2, appendix pp9-10), the predictor selection process was merely data-driven that emphasized achieving higher performance metrics but paid less attention to the empirical claims of fundamental mechanisms, which might cause potential bias in real clinical settings. 33 Besides, we tried to transparentise the employed ML model by visually interpreting with SHAP plots; however, it can still not be fully explained with the exact extent to which it can affect the model and impact the prediction outcomes, leading to misused applications under certain circumstances. 34 , 35 Last, the model we established was based on data from UK-Biobank where the anticipated individuals were mainly of White ethnicity and European ancestry, and the proposed model has not been adequately validated in other cohorts.…”

Section: Discussionmentioning

confidence: 99%

Development of a novel dementia risk prediction model in the general population: A large, longitudinal, population-based machine-learning study

et al. 2022

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Development of a novel dementia risk prediction model in the general population: A large, longitudinal, population-based machine-learning study

et al. 2022

View full text Add to dashboard Cite

“…Ethical concerns regarding the use of AI to analyze large amounts of health care data have also been raised in the literature. In establishing a research ethics framework for health care ML, McCraden et al [ 196 , 197 ] note how AI can influence 2 phases of health care research: hypothesis generation and hypothesis testing. AI research focused on hypothesis generation applies computational techniques to large data sets to explore models with potential clinical applicability [ 197 ].…”

Section: Discussionmentioning

confidence: 99%

“…In establishing a research ethics framework for health care ML, McCraden et al [ 196 , 197 ] note how AI can influence 2 phases of health care research: hypothesis generation and hypothesis testing. AI research focused on hypothesis generation applies computational techniques to large data sets to explore models with potential clinical applicability [ 197 ]. This type of exploratory research raises important ethical issues, such as the protection of data privacy and tensions between enabling ready access to data and the requirements of informed consent [ 197 ].…”

Section: Discussionmentioning

confidence: 99%

“…AI research focused on hypothesis generation applies computational techniques to large data sets to explore models with potential clinical applicability [ 197 ]. This type of exploratory research raises important ethical issues, such as the protection of data privacy and tensions between enabling ready access to data and the requirements of informed consent [ 197 ]. Most articles from our review under this theme fit into the hypothesis generation phase and used AI for exploratory research on the determinants of health outcomes in oncology.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The Impact of Artificial Intelligence on Health Equity in Oncology: Scoping Review

Istasy¹,

Lee²,

Iansavichene³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background The field of oncology is at the forefront of advances in artificial intelligence (AI) in health care, providing an opportunity to examine the early integration of these technologies in clinical research and patient care. Hope that AI will revolutionize health care delivery and improve clinical outcomes has been accompanied by concerns about the impact of these technologies on health equity. Objective We aimed to conduct a scoping review of the literature to address the question, “What are the current and potential impacts of AI technologies on health equity in oncology?” Methods Following PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines for scoping reviews, we systematically searched MEDLINE and Embase electronic databases from January 2000 to August 2021 for records engaging with key concepts of AI, health equity, and oncology. We included all English-language articles that engaged with the 3 key concepts. Articles were analyzed qualitatively for themes pertaining to the influence of AI on health equity in oncology. Results Of the 14,011 records, 133 (0.95%) identified from our review were included. We identified 3 general themes in the literature: the use of AI to reduce health care disparities (58/133, 43.6%), concerns surrounding AI technologies and bias (16/133, 12.1%), and the use of AI to examine biological and social determinants of health (55/133, 41.4%). A total of 3% (4/133) of articles focused on many of these themes. Conclusions Our scoping review revealed 3 main themes on the impact of AI on health equity in oncology, which relate to AI’s ability to help address health disparities, its potential to mitigate or exacerbate bias, and its capability to help elucidate determinants of health. Gaps in the literature included a lack of discussion of ethical challenges with the application of AI technologies in low- and middle-income countries, lack of discussion of problems of bias in AI algorithms, and a lack of justification for the use of AI technologies over traditional statistical methods to address specific research questions in oncology. Our review highlights a need to address these gaps to ensure a more equitable integration of AI in cancer research and clinical practice. The limitations of our study include its exploratory nature, its focus on oncology as opposed to all health care sectors, and its analysis of solely English-language articles.

show abstract

“…To mitigate these risks, several AI implementation pathways have been described ( 2 , 3 ). We have previously outlined a 3-stage roadmap for the evaluation and validation of AI models into clinical care ( 4 , 5 ), which has been implemented at scale at our institution. These phases include (1) exploratory model development, (2) a silent trial, and (3) prospective clinical evaluation.…”

Section: Introductionmentioning

confidence: 99%

The silent trial - the bridge between bench-to-bedside clinical AI applications

Kwong

Erdman

Khondker

et al. 2022

Front. Digit. Health

Self Cite

View full text Add to dashboard Cite

As more artificial intelligence (AI) applications are integrated into healthcare, there is an urgent need for standardization and quality-control measures to ensure a safe and successful transition of these novel tools into clinical practice. We describe the role of the silent trial, which evaluates an AI model on prospective patients in real-time, while the end-users (i.e., clinicians) are blinded to predictions such that they do not influence clinical decision-making. We present our experience in evaluating a previously developed AI model to predict obstructive hydronephrosis in infants using the silent trial. Although the initial model performed poorly on the silent trial dataset (AUC 0.90 to 0.50), the model was refined by exploring issues related to dataset drift, bias, feasibility, and stakeholder attitudes. Specifically, we found a shift in distribution of age, laterality of obstructed kidneys, and change in imaging format. After correction of these issues, model performance improved and remained robust across two independent silent trial datasets (AUC 0.85–0.91). Furthermore, a gap in patient knowledge on how the AI model would be used to augment their care was identified. These concerns helped inform the patient-centered design for the user-interface of the final AI model. Overall, the silent trial serves as an essential bridge between initial model development and clinical trials assessment to evaluate the safety, reliability, and feasibility of the AI model in a minimal risk environment. Future clinical AI applications should make efforts to incorporate this important step prior to embarking on a full-scale clinical trial.

show abstract

A Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning

Cited by 57 publications

References 70 publications

Development of a novel dementia risk prediction model in the general population: A large, longitudinal, population-based machine-learning study

Development of a novel dementia risk prediction model in the general population: A large, longitudinal, population-based machine-learning study

The Impact of Artificial Intelligence on Health Equity in Oncology: Scoping Review

The silent trial - the bridge between bench-to-bedside clinical AI applications

Contact Info

Product

Resources

About