Population Bias in Polygenic Risk Prediction Models for Coronary Artery Disease

Gola, Damian; Erdmann, Jeanette; Läll, Kristi; Mägi, Reedik; Müller‐Myhsok, Bertram; Schunkert, Heribert; König, Inke R.

doi:10.1161/circgen.120.002932

Cited by 34 publications

(25 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generalizability of a risk prediction model is an important criteria to establish reliable and safe use of the model, even under care settings different from the development cohort. A number of studies have reported lack of generalizability of risk prediction models across settings, such as different countries, [45][46][47][48] hospitals, 25,[49][50][51][52] or time periods. [53][54][55] For instance, Austin et al 56 study validity of mortality risk prediction models across geographies in terms of discrimination and calibration measures.…”

Section: Discussionmentioning

confidence: 99%

“…Generalizability of a risk prediction model is an important criteria to establish reliable and safe use of the model even under care settings different from the development cohort. A number of studies have reported lack of generalizability of risk prediction models across settings such as different countries (46)(47)(48)(49), hospitals (27,(50)(51)(52)(53), or time periods (54)(55)(56).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

Singh

Mhasawade

Chunara

2021

Preprint

View full text Add to dashboard Cite

Importance: Modern predictive models require large amounts of data for training and evaluation which can result in building models that are specific to certain locations, populations in them and clinical practices. Yet, best practices and guidelines for clinical risk prediction models have not yet considered such challenges to generalizability. Objectives: To investigate changes in measures of predictive discrimination, calibration, and algorithmic fairness when transferring models for predicting in-hospital mortality across ICUs in different populations. Also, to study the reasons for the lack of generalizability in these measures. Design, Setting, and Participants: In this multi-center cross-sectional study, electronic health records from 179 hospitals across the US with 70,126 hospitalizations were analyzed. Time of data collection ranged from 2014 to 2015. Main Outcomes and Measures: The main outcome is in-hospital mortality. Generalization gap, defined as difference between model performance metrics across hospitals, is computed for discrimination and calibration metrics, namely area under the receiver operating characteristic curve (AUC) and calibration slope. To assess model performance by race variable, we report differences in false negative rates across groups. Data were also analyzed using a causal discovery algorithm "Fast Causal Inference" (FCI) that infers paths of causal influence while identifying potential influences associated with unmeasured variables. Results: In-hospital mortality rates differed in the range of 3.9%-9.3% (1st-3rd quartile) across hospitals. When transferring models across hospitals, AUC at the test hospital ranged from 0.777 to 0.832 (1st to 3rd quartile; median 0.801); calibration slope from 0.725 to 0.983 (1st to 3rd quartile; median 0.853); and disparity in false negative rates from 0.046 to 0.168 (1st to 3rd quartile; median 0.092). When transferring models across geographies, AUC ranged from 0.795 to 0.813 (1st to 3rd quartile; median 0.804); calibration slope from 0.904 to 1.018 (1st to 3rd quartile; median 0.968); and disparity in false negative rates from 0.018 to 0.074 (1st to 3rd quartile; median 0.040). Distribution of all variable types (demography, vitals, and labs) differed significantly across hospitals and regions. Shifts in the race variable distribution and some clinical (vitals, labs and surgery) variables by hospital or region. Race variable also mediates differences in the relationship between clinical variables and mortality, by hospital/region. Conclusions and Relevance: Group-specific metrics should be assessed during generalizability checks to identify potential harms to the groups. In order to develop methods to improve and guarantee performance of prediction models in new environments for groups and individuals, better understanding and provenance of health processes as well as data generating processes by sub-group are needed to identify and mitigate sources of variation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

Singh

Mhasawade

Chunara

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…scored cohort) sets are drawn from different populations. 21 Fourth, the AGRE sample may be underpowered to detect weak PRS associations due to its small samplesize.…”

Section: Discussionmentioning

confidence: 99%

Exploring Polygenic Contributors To Subgroups of Comorbid Conditions In Autism Spectrum Disorder

Klein

D’Urso

Eapen

et al. 2021

Preprint

View full text Add to dashboard Cite

Individuals with autism spectrum disorder (ASD) have heterogeneous comorbid conditions. This study examined whether comorbid conditions in ASD are associated with polygenic risk scores (PRS) of ASD or PRS of comorbid conditions in non-ASD specific populations. Genome-wide single nucleotide polymorphism (SNP) data were obtained from 1,386 patients with ASD from the Autism Genetic Resource Exchange (AGRE) study. After excluding individuals with missing clinical information concerning comorbid conditions, a total of 707 patients were included in the study. A total of 18 subgroups of comorbid conditions (’topics’) were identified using a machine learning algorithm, topic modeling. PRS for ASD were computed using a genome-wide association meta-analysis of 18,381 cases and 27,969 controls. From these 18 topics, Topic 6 (over-represented by allergies) (p = 1.72 x 10−3) and Topic 17 (over-represented by sensory processing issues such as low pain tolerance) (p = .037) were associated with PRS of ASD. For associated topics, we further assessed their associations with PRS of their corresponding comorbid conditions based on non-ASD specific populations. However, these two topics were not associated with the PRS of allergies and chronic pain disorder, respectively. Characteristics of the present AGRE sample and those samples used in the original GWAS for ASD, allergies, and chronic pain disorder, may differ due to significant clinical heterogeneity that exists in the ASD population. Additionally, the AGRE sample may be underpowered and therefore insensitive to weak PRS associations due to a relatively small sample size. Findings imply that susceptibility genes of ASD may contribute more to the occurrence of allergies and sensory processing issues in individuals with ASD, compared with the susceptibility genes for their corresponding phenotypes. Since these comorbid conditions (i.e., allergies and pain sensation issues) cannot be attributable to the corresponding comorbidity-specific biological factors in non-ASD individuals, clinical management for these comorbid conditions may still depend on treatments for core symptoms of ASD.

show abstract

“…scored cohort) sets are drawn from different populations. [40] Fourth, the AGRE sample may be underpowered to detect weak PRS associations due to its small samplesize.…”

Section: Limitationsmentioning

confidence: 99%

Exploring Polygenic Contributors to Subgroups of Comorbid Conditions in Atism Spectrum Disorder

Klein

D’Urso

Eapen

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Individuals with autism spectrum disorder (ASD) have heterogeneous comorbid conditions. This study examined whether comorbid conditions in ASD are associated with polygenic risk scores (PRS) of ASD or PRS of comorbid conditions in non-ASD specific populations. Genome-wide single nucleotide polymorphism (SNP) data were obtained from 1,386 patients with ASD from the Autism Genetic Resource Exchange (AGRE) study. After excluding individuals with missing clinical information concerning comorbid conditions, a total of 707 patients were included in the study. Methods: A total of 18 subgroups of comorbid conditions (’topics’) were identified using a machine learning algorithm, topic modeling. PRS for ASD were computed using a genome-wide association meta-analysis of 18,381 cases and 27,969 controls. Results: From these 18 topics, Topic 6 (over-represented by allergies) ( p = 1.72x10 − 3 ) and Topic 17 (over-represented by sensory processing issues such as low pain tolerance) ( p = .037) were associated with PRS of ASD. For associated topics, we further assessed their associations with PRS of their corresponding comorbid conditions based on non-ASD specific populations. However, these two topics were not associated with the PRS of allergies and chronic pain disorder, respectively. Limitations: Characteristics of the present AGRE sample and those samples used in the original GWAS for ASD, allergies, and chronic pain disorder, may differ due to significant clinical heterogeneity that exists in the ASD population. Additionally, the AGRE sample may be underpowered and therefore insensitive to weak PRS associations due to low relatively small sample size. Conclusions: Findings imply that susceptibility genes of ASD may contribute more to the occurrence of allergies and sensory processing issues in individuals with ASD, compared with the susceptibility genes for their corresponding phenotypes. Since these comorbid conditions (i.e., allergies and pain sensation issues) cannot be attributable to the corresponding comorbidity-specific biological factors in non-ASD individuals, clinical management for these comorbid conditions may still depend on treatments for core symptoms of ASD.

show abstract

Population Bias in Polygenic Risk Prediction Models for Coronary Artery Disease

Cited by 34 publications

References 15 publications

Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

Exploring Polygenic Contributors To Subgroups of Comorbid Conditions In Autism Spectrum Disorder

Exploring Polygenic Contributors to Subgroups of Comorbid Conditions in Atism Spectrum Disorder

Contact Info

Product

Resources

About