A Distribution-based Method for Assessing The Differences between Clinical Trial Target Populations and Patient Populations in Electronic Health Records

Abstract

SummaryObjective: To improve the transparency of clinical trial generalizability and to illustrate the method using Type 2 diabetes as an example. Methods: Our data included 1,761 diabetes clinical trials and the electronic health records (EHR) of 26,120 patients with Type 2 diabetes who visited Columbia University Medical Center of NewYork Presbyterian Hospital. The two populations were compared using the Generalizability Index for Study Traits (GIST) on the earliest diagnosis age and the mean hemoglobin A 1c (HbA 1c ) values. Results: Greater than 70% of Type 2 diabetes studies allow patients with HbA 1c measures between 7 and 10.5, but less than 40% of studies allow HbA 1c <7 and fewer than 45% of studies allow HbA 1c >10.5. In the real-world population, only 38% of patients had HbA 1c between 7 and 10.5, with 12% having values above the range and 52% having HbA 1c <7. The GIST for HbA 1c was 0.51. Most studies adopted broad age value ranges, with the most common restrictions excluding patients >80 or <18 years. Most of the real-world population fell within this range, but 2% of patients were <18 at time of first diagnosis and 8% were >80. The GIST for age was 0.75. Conclusions: We contribute a scalable method to profile and compare aggregated clinical trial target populations with EHR patient populations. We demonstrate that Type 2 diabetes studies are more generalizable with regard to age than they are with regard to HbA 1c . We found that the generalizability of age increased from Phase 1 to Phase 3 while the generalizability of HbA 1c decreased during those same phases. This method can generalize to other medical conditions and other continuous or binary variables. We envision the potential use of EHR data for examining the generalizability of clinical trials and for defining population-representative clinical trial eligibility criteria.

Multivariate analysis of the population representativeness of related clinical studies

“…We can see that all the top 10 underrepresented population subgroups include elderly (> 64 years old) patients with relatively low HbA1c (less than 7.2%) and a wide range of BMI (18.5 kg/m 2 –39 kg/m 2 ). This finding is consistent with our population representativeness analysis using one variable at a time [17, 21]. Compared with underrepresented subgroups, the well-represented patients (eligible for >= 66.4% of trials) are younger (age between 44 and 64) and have higher HbA1c (> 7.2%).…”

“…This metric, called Generalizability Index for Study Traits (GIST) [17], allows quantification of collective a priori generalizability of multiple studies using one study trait at a time. Since population representativeness is a multi-dimensional problem [18] involving multiple criteria, it is not informative to apply GIST on multiple variables linearly one after another for assessing multivariate population representativeness.…”

“…Thus, we are motivated to extend GIST in a multivariate setting. Built upon our previous work [17, 19–22], this paper presents a framework for multivariate analysis of population representativeness of related clinical studies. The contribution of this work is two-fold: (1) a quantitative measure of a priori generalizability incorporating multiple numeric eligibility criteria variables (referred to as “mGIST” from this point on); and (2) a method called MAGIC for identifying and characterizing underrepresented population subgroups with combinations of value intervals across multiple eligibility criteria variables.…”

Prediction of black box warning by mining patterns of Convergent Focus Shift in clinical trial study populations using linked public data

“…We assessed the popularity of a drug based on its retail sales in US by obtaining this information from drugs.com [17]. We identified 402 drugs that appeared at least once in the top-selling lists during these 11 years, including 200 drugs hitting the list between 2003 and 2013 and 100 drugs between 2011 and 2013.…”

Assessing the population representativeness of colorectal cancer treatment clinical trials

“…To quantify a priori generalizability, Weng et al previously introduced a quantitative metric that compares the study population of the trials, derived from their eligibility criteria, with the real world patients who would benefit from the trial results [6]. The metric, Generalizability Index on Study Traits (GIST), can quantify the population representativeness of a set of clinical trials with respect to a single quantitative variable such as age and BMI.…”