Performance of the Framingham risk models and pooled cohort equations for predicting 10-year risk of cardiovascular disease: a systematic review and meta-analysis

Abstract

Background: The Framingham risk models and pooled cohort equations (PCE) are widely used and advocated in guidelines for predicting 10-year risk of developing coronary heart disease (CHD) and cardiovascular disease (CVD) in the general population. Over the past few decades, these models have been extensively validated within different populations, which provided mounting evidence that local tailoring is often necessary to obtain accurate predictions. The objective is to systematically review and summarize the predictive performance of three widely advocated cardiovascular risk prediction models (Framingham Wilson 1998, Framingham ATP III 2002 and PCE 2013 in men and women separately, to assess the generalizability of performance across different subgroups and geographical regions, and to determine sources of heterogeneity in the findings across studies. Methods: A search was performed in October 2017 to identify studies investigating the predictive performance of the aforementioned models. Studies were included if they externally validated one or more of the original models in the general population for the same outcome as the original model. We assessed risk of bias for each validation and extracted data on population characteristics and model performance. Performance estimates (observed versus expected (OE) ratio and c-statistic) were summarized using a random effects models and sources of heterogeneity were explored with meta-regression. Results: The search identified 1585 studies, of which 38 were included, describing a total of 112 external validations. Results indicate that, on average, all models overestimate the 10-year risk of CHD and CVD (pooled OE ratio ranged from 0.58 (95% CI 0.43-0.73; Wilson men) to 0.79 (95% CI 0.60-0.97; ATP III women)). Overestimation was most pronounced for high-risk individuals and European populations. Further, discriminative performance was better in women for all models. There was considerable heterogeneity in the c-statistic between studies, likely due to differences in population characteristics. Conclusions: The Framingham Wilson, ATP III and PCE discriminate comparably well but all overestimate the risk of developing CVD, especially in higher risk populations. Because the extent of miscalibration substantially varied across settings, we highly recommend that researchers further explore reasons for overprediction and that the models be updated for specific populations.

Machine learning prediction in cardiovascular diseases: a meta-analysis

“…These traditional CVD risk scores have several limitations, including variations among validation cohorts, particularly in specific populations such as patients with rheumatoid arthritis 19,20 . Under some circumstances, the Framingham score overestimates CVD risk, potentially leading to overtreatment 20 . In general, these risk scores encompass a limited number of predictors and omit several important variables.…”

External validation of three atherosclerotic cardiovascular disease risk equations in rural areas of Xinjiang, China

“…More details can be seen in Additional file 1: Table S5. Improved calibration is of vital importance to risk classification and evaluation of net benefit, so recalibration can be a useful method to tailor and generalize a risk equation to an external population [41].…”

Cardiovascular risk in patients with severe mental illness in Italy

“…First, although conducted in Asian, European, and North American countries, they all employed the Framingham risk score algorithm, which was validated on the population of Framingham, USA, several decades ago. In recent years, there has been mounting evidence that the Framingham model overestimates the risk of developing CV disease, especially in high-risk individuals and European populations [9]. Accordingly, risk scores in patients and controls should be calculated employing an algorithm of risk validated on the reference population.…”