Conditional distribution modeling as an alternative method for covariates simulation: Comparison with joint multivariate normal and bootstrap techniques

Smania, Giovanni; Jonsson, E. Niclas

doi:10.1002/psp4.12613

Cited by 10 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In traditional parametric methods such as linear regression, each covariate can be interpreted clearly (e.g., for each 1 increase in x, we observe 2 increases in y) [ 17 , 49 ]. However, due to the complexity of the non-parametric algorithms that are common in machine-learning methods, it is impossible for a human to analyze each tree and execute an explanation of how the machine-learning method works [ 1 , 62 – 65 ]. Thus, using SHAP allows for a similar covariate interpretation as linear regression even if the exact effect-sizes of the covariates cannot be interpreted the way it can in linear regression [ 15 , 22 , 49 , 66 – 68 ].…”

Section: Discussionmentioning

confidence: 99%

“…to the complexity of the non-parametric algorithms that are common in machine-learning methods, it is impossible for a human to analyze each tree and execute an explanation of how the machine-learning method works [1,[62][63][64][65]. Thus, using SHAP allows for a similar covariate interpretation as linear regression even if the exact effect-sizes of the covariates cannot be interpreted the way it can in linear regression [15,22,49,[66][67][68]…”

Section: Plos Onementioning

confidence: 99%

See 1 more Smart Citation

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Huang

2023

PLoS ONE

View full text Add to dashboard Cite

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Plos Onementioning

confidence: 99%

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Huang

2023

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…With such computational throughput, clinical study designs can be optimized, with mechanistic insight into formulation performance being accounted for in minutes rather than hours. We also point out that since the framework relies on a relatively small number of uncorrelated parameters, the population parameters can be efficiently sampled without recourse to constrained sampling , to ensure physically reasonable individuals.…”

Section: Discussionmentioning

confidence: 99%

Lilly Absorption Modeling Platform: A Tool for Early Absorption Assessment

Stamatis

Rose

2021

Mol. Pharmaceutics

View full text Add to dashboard Cite

Oral drug absorption modeling has developed at a rapid pace in the 40 years or so since the first ideas for mathematical approaches to oral absorption were introduced. The success of compartmental approaches accelerated the uptake of absorption modeling, and over the last 20 years, work on absorption modeling has shifted almost exclusively to the compartmental framework. This report describes a new noncompartmental absorption modeling framework, the Lilly Absorption Modeling Platform (LAMP). LAMP connects a well-mixed stomach to a continuous tube model of the small intestine with plug flow. Within the continuous tube framework, the model includes intestinal mixing and a novel highly tunable precipitation model that can describe a combination of rapid nucleation and slow growth. The framework is designed to balance speed, consistency, and ease of use with a minimum of model complexity to capture the essential features of gastrointestinal (GI) physiology and critical elements of the oral absorption process. The model was validated based on predictions of the fraction absorbed and the maximum absorbable dose for a set of Eli Lilly and Company clinical compounds.

show abstract

“…15 The CD approach, which uses a multiple imputation algorithm to iteratively impute covariate values for virtual patients, was used as implemented by the developers of the method. 16 The standard multiple imputation method "predictive mean matching" was used, corresponding to their paper. The distribution-based methods used were the MVND and marginal distributions (MDs), through maximum likelihood estimation.…”

Section: Evaluation Of Simulation Performancementioning

confidence: 99%

“…However, these methods are only able to simulate patients that are already present in the data set and require a large enough number of patients to be included. These shortcomings were addressed by a recently proposed imputation method using conditional distributions (CDs), 16 although this method remains dependent on access to patient‐level data. Distribution‐based simulation methods for virtual patient simulation do not require patient‐level data access.…”

Section: Figurementioning

confidence: 99%

Virtual Patient Simulation Using Copula Modeling

Zwep,

Guo,

Nagler

et al. 2023

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Virtual patient simulation is increasingly performed to support model‐based optimization of clinical trial designs or individualized dosing strategies. Quantitative pharmacological models typically incorporate individual‐level patient characteristics, or covariates, which enable the generation of virtual patient cohorts. The individual‐level patient characteristics, or covariates, used as input for such simulations should accurately reflect the values seen in real patient populations. Current methods often make unrealistic assumptions about the correlation between patient's covariates or require direct access to actual data sets with individual‐level patient data, which may often be limited by data sharing limitations. We propose and evaluate the use of copulas to address current shortcomings in simulation of patient‐associated covariates for virtual patient simulations for model‐based dose and trial optimization in clinical pharmacology. Copulas are multivariate distribution functions that can capture joint distributions, including the correlation, of covariate sets. We compare the performance of copulas to alternative simulation strategies, and we demonstrate their utility in several case studies. Our work demonstrates that copulas can reproduce realistic patient characteristics, both in terms of individual covariates and the dependence structure between different covariates, outperforming alternative methods, in particular when aiming to reproduce high‐dimensional covariate sets. In conclusion, copulas represent a versatile and generalizable approach for virtual patient simulation which preserve relationships between covariates, and offer an open science strategy to facilitate re‐use of patient data sets.

show abstract

Conditional distribution modeling as an alternative method for covariates simulation: Comparison with joint multivariate normal and bootstrap techniques

Abstract: This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

Cited by 10 publications

References 25 publications

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Lilly Absorption Modeling Platform: A Tool for Early Absorption Assessment

Virtual Patient Simulation Using Copula Modeling

Contact Info

Product

Resources

About