Multi-cohort modeling strategies for scalable globally accessible prostate cancer risk tools

Abstract

Background Online clinical risk prediction tools built on data from multiple cohorts are increasingly being utilized for contemporary doctor-patient decision-making and validation. This report outlines a comprehensive data science strategy for building such tools with application to the Prostate Biopsy Collaborative Group prostate cancer risk prediction tool. Methods We created models for high-grade prostate cancer risk using six established risk factors. The data comprised 8492 prostate biopsies collected from ten institutions, 2 in Europe and 8 across North America. We calculated area under the receiver operating characteristic curve (AUC) for discrimination, the Hosmer-Lemeshow test statistic (HLS) for calibration and the clinical net benefit at risk threshold 15%. We implemented several internal cross-validation schemes to assess the influence of modeling method and individual cohort on validation performance. Results High-grade disease prevalence ranged from 18% in Zurich (1863 biopsies) to 39% in UT Health San Antonio (899 biopsies). Visualization revealed outliers in terms of risk factors, including San Juan VA (51% abnormal digital rectal exam), Durham VA (63% African American), and Zurich (2.8% family history). Exclusion of any cohort did not significantly affect the AUC or HLS, nor did the choice of prediction model (pooled, random-effects, meta-analysis). Excluding the lowest-prevalence Zurich cohort from training sets did not statistically significantly change the validation metrics for any of the individual cohorts, except for Sunnybrook, where the effect on the AUC was minimal. Therefore the final multivariable logistic model was built by pooling the data from all cohorts using logistic regression. Higher prostate-specific antigen and age, abnormal digital rectal exam, African ancestry and a family history of prostate cancer increased risk of high-grade prostate cancer, while a history of a prior negative prostate biopsy decreased risk (all p-values < 0.004). Conclusions We have outlined a multi-cohort model-building internal validation strategy for developing globally accessible and scalable risk prediction tools.

Accommodating heterogeneous missing data patterns for prostate cancer risk prediction

“…The 3 rd cohort separated by the black vertical line is used as an external validation set, and leave-one-cohort-out cross-validation was applied to the other cohorts. The 10 cohorts used for training the model followed the PBCG prospective protocol in data collection, whereas the external validation cohort supplied retrospective data from a single institution that performs a high annual number of prostate biopsies to the PBCG [2,3]. Included data came from patients who had received a prostate biopsy following a PSA test under local standard-of-care and may be seen as representative of patients in North America, including Puerto Rico, and Europe.…”

“…For developing the prior tool, multiple methods for aggregating clinical data on a small number of variables across heterogeneous centers comprising different risk factor distributions and risk factor-outcome associations were compared. The simplest approach of pooling individual-level data and fitting a multiple logistic regression model proved to be most accurate [3]. The resulting risk calculator was published online at riskcalc.…”

“…Department of Life Sciences, Technical University of Munich, Freising, Germany 2. Department of Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, OH, USA 3. Section of Urology, Durham Veterans Administration Health Care System, Durham, NC, USA 4.…”

Globally accessible end-user-friendly prostate cancer risk prediction tools based on contemporary cohorts with heterogeneous missing risk factors

“…The 10 cohorts used for training the model followed the PBCG prospective protocol in data collection, whereas the external validation cohort supplied retrospective data from a single institution that performs a high annual number of prostate biopsies to the PBCG [2,3]. Included data came from patients who had received a prostate biopsy following a PSA test under local standard-of-care and may be seen as representative of patients in North America, including Puerto Rico, and Europe.…”

“…For developing the prior tool, multiple methods for aggregating clinical data on a small number of variables across heterogeneous centers comprising different risk factor distributions and risk factor-outcome associations were compared. The simplest approach of pooling individual-level data and tting a multiple logistic regression model proved to be most accurate [3]. The resulting risk calculator was published online at riskcalc.org to facilitate its use in daily routine.…”