MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care

Hernandez‐Boussard, Tina; Bozkurt, Selen; Ioannidis, John P. A.; Shah, Nigam H.

doi:10.1093/jamia/ocaa088

Cited by 194 publications

(113 citation statements)

References 21 publications

(11 reference statements)

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…This study’s findings highlight the necessity of evaluating population representativeness and potential biases. 30 The transparency in such information provides the end user with an opportunity to mitigate such biases and appropriately deploy models across populations.…”

Section: Discussionmentioning

confidence: 99%

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

et al. 2021

Self Cite

View full text Add to dashboard Cite

IMPORTANCE Randomized clinical trials (RCTs) are considered the criterion standard for clinical evidence. Despite their many benefits, RCTs have limitations, such as costliness, that may reduce the generalizability of their findings among diverse populations and routine care settings. OBJECTIVE To assess the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic castration-resistant prostate cancer (CRPC) when the model is applied to real-world data from electronic health records (EHRs). DESIGN, SETTING, AND PARTICIPANTS The RCT-trained model and patient data from the RCTs were obtained from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge for prostate cancer, which occurred from March 16 to July 27, 2015. This challenge included 4 phase 3 clinical trials of patients with metastatic CRPC. Real-world data were obtained from the EHRs of a tertiary care academic medical center that includes a comprehensive cancer center. In this study, the DREAM challenge RCT-trained model was applied to real-world data from January 1, 2008, to December 31, 2019; the model was then retrained using EHR data with optimized feature selection. Patients with metastatic CRPC were divided into RCT and EHR cohorts based on data source. Data were analyzed from March 23, 2018, to October 22, 2020. EXPOSURES Patients who received treatment for metastatic CRPC. MAIN OUTCOMES AND MEASURES The primary outcome was the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic CRPC when the model is applied to real-world data. Model performance was compared using 10-fold cross-validation according to time-dependent integrated area under the curve (iAUC) statistics. RESULTS Among 2113 participants with metastatic CRPC, 1600 participants were included in the RCT cohort, and 513 participants were included in the EHR cohort. The RCT cohort comprised a larger proportion of White participants (1390 patients [86.9%] vs 337 patients [65.7%]) and a smaller proportion of Hispanic participants (14 patients [0.9%] vs 42 patients [8.2%]), Asian participants (41 patients [2.6%] vs 88 patients [17.2%]), and participants older than 75 years (388 patients [24.3%] vs 191 patients [37.2%]) compared with the EHR cohort. Participants in the RCT cohort also had fewer comorbidities (mean [SD], 1.6 [1.8] comorbidities vs 2.5 [2.6] comorbidities, respectively) compared with those in the EHR cohort. Of the 101 variables used in the RCT-derived model, 10 were not available in the EHR data set, 3 of which were among the top 10 features in the DREAM challenge RCT model. The best-performing EHR-trained model included only 25 of the 101 variables included in the RCT-trained model. The performance of the RCT-trained and EHR-trained models was adequate in the EHR cohort (mean [SD] iAUC, 0.722 [0.118] and 0.762 [0.106], respectively); model optimization was associated with improved performance of the best-performing EHR model (mean [SD] iAUC, (continued) Key Points ...

show abstract

Section: Discussionmentioning

confidence: 99%

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…From a technical perspective, candidate algorithms and tools should be validated at other sites, account for differential performance in subgroups, and explicitly report the uncertainty around any estimates or recommendations 78 . Furthermore, papers describing model development and performance assessments should adhere to reporting standards for transparency and provide important information about validity, biases, and generalizability to other settings 79 . Once high-quality AI solutions are developed, additional factors beyond performance must be considered to increase the likelihood of successful implementation and adoption by individual providers.…”

Section: Discussionmentioning

confidence: 99%

The potential of artificial intelligence to improve patient safety: a scoping review

et al. 2021

View full text Add to dashboard Cite

Artificial intelligence (AI) represents a valuable tool that could be used to improve the safety of care. Major adverse events in healthcare include: healthcare-associated infections, adverse drug events, venous thromboembolism, surgical complications, pressure ulcers, falls, decompensation, and diagnostic errors. The objective of this scoping review was to summarize the relevant literature and evaluate the potential of AI to improve patient safety in these eight harm domains. A structured search was used to query MEDLINE for relevant articles. The scoping review identified studies that described the application of AI for prediction, prevention, or early detection of adverse events in each of the harm domains. The AI literature was narratively synthesized for each domain, and findings were considered in the context of incidence, cost, and preventability to make projections about the likelihood of AI improving safety. Three-hundred and ninety-two studies were included in the scoping review. The literature provided numerous examples of how AI has been applied within each of the eight harm domains using various techniques. The most common novel data were collected using different types of sensing technologies: vital sign monitoring, wearables, pressure sensors, and computer vision. There are significant opportunities to leverage AI and novel data sources to reduce the frequency of harm across all domains. We expect AI to have the greatest impact in areas where current strategies are not effective, and integration and complex analysis of novel, unstructured data are necessary to make accurate predictions; this applies specifically to adverse drug events, decompensation, and diagnostic errors.

show abstract

“…Race distinguishes individuals based on ancestry and combinations of physical characteristics, whereas ethnicity focuses on behavior and culture in addition to physical features [ 77 ]. Inconsistent reporting of ethnic and racial information hinders the ability to perform meta-analyses across multiple data sets and may limit ethnoracial equity in future AI applications [ 78 ]. In their writings on eliminating health disparities, Fremont and Lurie state that data pertaining to race and ethnicity are collected by a variety of sources, but “the utility of these data is constrained by ongoing problems with reliability, completeness, and lack of comparability across data sources” [ 79 ].…”

Section: Discussionmentioning

confidence: 99%

The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations

et al. 2021

View full text Add to dashboard Cite

There is clear evidence to suggest that diabetes does not affect all populations equally. Among adults living with diabetes, those from ethnoracial minority communities—foreign-born, immigrant, refugee, and culturally marginalized—are at increased risk of poor health outcomes. Artificial intelligence (AI) is actively being researched as a means of improving diabetes management and care; however, several factors may predispose AI to ethnoracial bias. To better understand whether diabetes AI interventions are being designed in an ethnoracially equitable manner, we conducted a secondary analysis of 141 articles included in a 2018 review by Contreras and Vehi entitled “Artificial Intelligence for Diabetes Management and Decision Support: Literature Review.” Two members of our research team independently reviewed each article and selected those reporting ethnoracial data for further analysis. Only 10 articles (7.1%) were ultimately selected for secondary analysis in our case study. Of the 131 excluded articles, 118 (90.1%) failed to mention participants’ ethnic or racial backgrounds. The included articles reported ethnoracial data under various categories, including race (n=6), ethnicity (n=2), race/ethnicity (n=3), and percentage of Caucasian participants (n=1). Among articles specifically reporting race, the average distribution was 69.5% White, 17.1% Black, and 3.7% Asian. Only 2 articles reported inclusion of Native American participants. Given the clear ethnic and racial differences in diabetes biomarkers, prevalence, and outcomes, the inclusion of ethnoracial training data is likely to improve the accuracy of predictive models. Such considerations are imperative in AI-based tools, which are predisposed to negative biases due to their black-box nature and proneness to distributional shift. Based on our findings, we propose a short questionnaire to assess ethnoracial equity in research describing AI-based diabetes interventions. At this unprecedented time in history, AI can either mitigate or exacerbate disparities in health care. Future accounts of the infancy of diabetes AI must reflect our early and decisive action to confront ethnoracial inequities before they are coded into our systems and perpetuate the very biases we aim to eliminate. If we take deliberate and meaningful steps now toward training our algorithms to be ethnoracially inclusive, we can architect innovations in diabetes care that are bound by the diverse fabric of our society.

show abstract

MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care

Cited by 194 publications

References 21 publications

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

The potential of artificial intelligence to improve patient safety: a scoping review

The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations

Contact Info

Product

Resources

About