Replication of Published Health Economic Obesity Models: Assessment of Facilitators, Hurdles and Reproduction Success

Abstract

Objectives This research aims to (1) replicate published health economic models, (2) compare reproduced results with original results, (3) identify facilitators and hurdles to model replicability and determine reproduction success, and (4) suggest model replication reporting standards to enhance model reproducibility, in the context of health economic obesity models. Methods Four health economic obesity models simulating an adult UK population were identified, selected for replication, and evaluated using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Reproduction results were compared to original results, focusing on cost-effectiveness outcomes, and the resulting reproduction success was assessed by published criteria. Replication facilitators and hurdles were identified and transferred into related reporting standards. Results All four case studies were state-transition models simulating costs and quality-adjusted life-years (QALYs). Comparing original versus reproduction outcomes, the following deviation ranges were observed: costs − 3.9 to 16.1% (mean over all model simulations 3.78%), QALYs − 3.7 to 2.1% (mean − 0.11%), and average cost-utility ratios − 3.0 to 17.9% (mean 4.28%). Applying different published criteria, an overall reproduction success was observed for three of four models. Key replication facilitators were input data tables and model diagrams, while missing standard deviations and missing formulas for equations were considered as key hurdles. Conclusions This study confirms the feasibility of rebuilding health economic obesity models, but minor to major assumptions were needed to fill reporting gaps. Model replications can help to assess the quality of health economic model documentation and can be used to validate current model reporting practices. Simple changes to actual CHEERS reporting criteria may solve identified replication hurdles.

Does the Structure Matter? An External Validation and Health Economic Results Comparison of Event Simulation Approaches in Severe Obesity

“…As basis for this research, three previously replicated obesity models were used [9][10][11][12][13]. These models reflect three main structural obesity event simulation approaches commonly used in health economic obesity modelling [8].…”

“…These models were selected on the basis of a previously published systematic review by our research group [ 8 ], and on the basis of minimal quality requirements based on an expert consensus [ 12 ]. All models were previously successfully replicated in TreeAge Pro (Version 2021 R1.1) on the basis of the published data [ 13 ]. For assessing the success of the model replications, we applied different criteria as defined and proposed in a recently published review on this topic [ 17 ].…”

Reproducibility and Scientific Integrity of Big Data Research in Urban Public Health and Digital Epidemiology: A Call to Action

“…Simulation approaches built upon reproducible research allow for the construction of accurate prediction models with important implications for healthcare [ 83 ] and public health [ 84 ]. In the same way, reproduction and replication of results for model validation are essential [ 85 , 86 , 87 ].…”

Testicular fat deposition attenuates reproductive performance via decreased follicle-stimulating hormone level and sperm meiosis and testosterone synthesis in mouse

“…Obesity is a prevalent issue worldwide and is a significant public health concern [ 1 , 2 ]. It has been demonstrated that obesity can have adverse effects on mammalian reproductive capacity [ 1 , 3 , 4 ].…”