Established patterns of animal study design undermine translation of disease-modifying therapies for Parkinson’s disease

Abstract

Translation of disease-modifying therapies in neurodegenerative disease has been disappointing. Parkinson’s disease (PD) was used to compare patterns of preclinical study design for symptomatic and potentially disease-modifying interventions. We examined the relationship of model, intervention type and timing, outcomes and outcome measures in 543 animal and human studies (1973–2015) across a contemporary cohort of animal and human interventional studies (n = 445), animal studies for approved interventions (n = 28), animal and human studies for those that failed to translate (n = 70). Detailed study design data were collected for 216 studies in non-human primate (NHP) and rodent toxin-induced models. Species-specific patterns of study design prevailed regardless of whether interventions were symptomatic or potentially disease-modifying. In humans and NHPs, interventions were typically given to both sexes well after the PD phenotype was established, and clinical outcome measures were collected at single (symptomatic) or multiple (disease-modifying) time-points. In rodents, interventions often preceded induction of the model, acute toxic protocols were common, usually given to young males, clinical outcome measures were used less commonly, and outcomes were less commonly assessed at multiple time points. These patterns were more prevalent in mice than rats. In contrast, study design factors such as randomization and blinding did not differ appreciably across symptomatic and disease-modifying intervention categories. The translational gap for potentially disease-modifying interventions in PD in part results from study designs, particularly in mice, that fail to model the progressive nature and relatively late intervention characteristic of PD, or that anchor mechanistic and neuropathologic data to longitudinal clinical outcomes. Even if measures to improve reproducibility are broadly adopted, perpetuation of these norms will continue to impede effective translation.

Menagerie: A text-mining tool to support animal-human translation in neurodegeneration research

“…However, PD confronts the same roadblocks as other neurodegenerative diseases in development of disease altering therapies [46,47]. We demonstrate retrieval of the expected patterns of animal species and model selection [48,49] for PD, and confirm previously identified [22] patterns of animal model use. This study paves the way for large scale evaluation of the entire PD corpus to identify those therapeutic approaches that have potential for clinical translation.…”

“…We used a text mining approach to extract characteristics that we have previously determined, using manual curation, to be reliably present in abstracts, and to be translationally useful [22,50]. These comprise information regarding therapeutic intervention, molecular target (s) or genes, species, model, overall outcome of the study and whether functionally relevant outcome measures [20] were reported.…”

“…This dataset consists of two parts. The training set (406 abstracts), developed in an earlier study [22], spans years 2015-2016 and was used for development and Fig 1. A system overview of the text-mining tool. Modules (species, models, genes, interventions and outcomes) defining types of data returned (far right) from an abstract (far left) are shown on the first row.…”

“…Several factors that undermine the reliability of animal studies have been identified. These include insufficient rigor in animal study design and reporting [6,17], publication bias [10,12], over-reporting of significance [18,19], over-reliance on non-clinical outcome measures [20], and entrenched use of certain model systems [3,21,22]. Together, these issues contribute to poor reproducibility of animal studies, and certainly worsen the translational gap [23].…”

“…Additionally, study design choices that influence generalizability extend beyond those needed to ensure unbiased study design. These are more contextually defined and concern the relationship between variables such as model choice [34] and mechanism of the intervention [22], integration of biomarker data with clinically relevant outcome measures, and use of progressive disease models and longitudinal study designs if neuroprotection is claimed. Addressing the potential generalizability of interventional animal studies is the impetus for the methods described in this paper.…”