External Validation of a Mammography-Derived AI-Based Risk Model in a U.S. Breast Cancer Screening Cohort of White and Black Women

Abstract

Despite the demonstrated potential of artificial intelligence (AI) in breast cancer risk assessment for personalizing screening recommendations, further validation is required regarding AI model bias and generalizability. We performed external validation on a U.S. screening cohort of a mammography-derived AI breast cancer risk model originally developed for European screening cohorts. We retrospectively identified 176 breast cancers with exams 3 months to 2 years prior to cancer diagnosis and a random sample of 4963 controls from women with at least one-year negative follow-up. A risk score for each woman was calculated via the AI risk model. Age-adjusted areas under the ROC curves (AUCs) were estimated for the entire cohort and separately for White and Black women. The Gail 5-year risk model was also evaluated for comparison. The overall AUC was 0.68 (95% CIs 0.64–0.72) for all women, 0.67 (0.61–0.72) for White women, and 0.70 (0.65–0.76) for Black women. The AI risk model significantly outperformed the Gail risk model for all women p < 0.01 and for Black women p < 0.01, but not for White women p = 0.38. The performance of the mammography-derived AI risk model was comparable to previously reported European validation results; non-significantly different when comparing White and Black women; and overall, significantly higher than that of the Gail model.

Concordant and Discordant Breast Density Patterns by Different approaches for Assessing Breast Density and Breast Cancer Risk

“…The modi ed Gail model has allowed for risk estimates for Asian-American women [29], albeit its C-statistics is low and has been estimated to be equal to 0.54 in an external validation study for Korean 40 229 women [30]. According to an external validation study in a U.S. screening cohort, the predictive performance of a mammography-derived AI risk model was signi cantly higher than that of the Gail model (0.68 vs. 0.55, p < 0.01) [31]. The limited discriminatory accuracy might be attributed to recent increase in the incidence of breast cancer in Korea;…”

European validation of an image-derived AI-based short-term risk model for individualized breast cancer screening—a nested case-control study

“… 36 It could be that image-based risk models benefit from adapting to specific screening settings. 38 , 39 , 40…”

Enhancing Accuracy in Breast Density Assessment Using Deep Learning: A Multicentric, Multi-Reader Study

“…However, the classification system has faced challenges due to the significant interobserver variability among radiologists, leading to inconsistencies and uncertainties in assessments [5][6][7]. Recent advancements in artificial intelligence (AI) and deep learning (DL) have demonstrated the potential to improve diagnostic accuracy in medical imaging [8][9][10]. This study investigates the efficacy of a deep learning-enhanced computer-aided diagnosis (CAD) system in evaluating breast tissue density according to the BI-RADS density classification.…”

“…Recent advancements in artificial intelligence (AI) and deep learning (DL) have demonstrated the potential to improve diagnostic accuracy in medical imaging [ 8 , 9 , 10 ]. This study investigates the efficacy of a deep learning-enhanced computer-aided diagnosis (CAD) system in evaluating breast tissue density according to the BI-RADS density classification.…”