Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation

Abstract

Comparisons between observed data and model simulations represent a critical component for establishing confidence in population physiologically-based pharmacokinetic (Pop-PBPK) models. Numerical predictive checks (NPC) that assess the proportion of observed data that correspond to Pop-PBPK model prediction intervals (PIs) are frequently used to qualify such models. We evaluated the effects of 3 components on the performance of NPC for qualifying Pop-PBPK model concentration-time predictions: (1) correlations (multiple samples per subject), (2) residual error, and (3) discrepancies in the distribution of demographics between observed and virtual subjects. Using a simulation-based study design, we artificially created observed pharmacokinetic (PK) datasets and compared them to model simulations generated under the same Pop-PBPK model. Observed datasets containing uncorrelated and correlated observations (+/− residual error) were formulated using different random-sampling techniques. In addition, we created observed datasets where the distribution of subject body weights differed from that of the virtual population used to generate model simulations. NPC for each observed dataset were computed based on the Pop-PBPK model's 90% PI. NPC were associated with inflated type-I-error rates (>0.10) for observed datasets that contained correlated observations, residual error, or both. Additionally, the performance of NPC were sensitive to the demographic distribution of observed subjects. Acceptable use of NPC was only demonstrated for the idealistic case where observed data were uncorrelated, free of residual error, and the demographic distribution of virtual subjects matched that of observed subjects. Considering the restricted applicability of NPC for Pop-PBPK model evaluation, their use in this context should be interpreted with caution.

In vitro–in vivo extrapolation of bexarotene metabolism in the presence of chronic kidney disease and acute kidney injury in rat using physiologically based pharmacokinetic modeling and extrapolation to human

“…Also, the observed PK levels were overlaid over the resulting 95% predictive interval (PI) in such populations (Maharaj et al., 2019). Then, the percentage of the observed data points that are outside the PI limits was calculated according to Equation (13): $$$\frac{1}{\mathrm{N}}\ast (\sum\limits _{\mathrm{i}=1}^{\mathrm{N}}{1}_{\left({\mathrm{Y}}_{\mathrm{i},\mathrm{t}}^{\mathrm{o}\mathrm{b}\mathrm{s}}< {\mathrm{Y}}_{2.5,\mathrm{t}}^{\mathrm{s}\mathrm{i}\mathrm{m}}\right)}+\sum\limits _{\mathrm{i}=1}^{\mathrm{N}}{1}_{\left({\mathrm{Y}}_{\mathrm{i},\mathrm{t}}^{\mathrm{o}\mathrm{b}\mathrm{s}}< {\mathrm{Y}}_{97.5,\mathrm{t}}^{\mathrm{s}\mathrm{i}\mathrm{m}}\right)\,})\,\ast \,100\%$$$ …”

“…Also, the observed PK levels were overlaid over the resulting 95% predictive interval (PI) in such populations (Maharaj et al, 2019).…”

Physiologically‐based pharmacokinetic model for alectinib, ruxolitinib, and panobinostat in the presence of cancer, renal impairment, and hepatic impairment

“…To evaluate the PopPBPK model‐predicted panobinostat plasma levels in cancer patients with RIP or HIP (Sharma, 2015; Slingerland, 2014), the resulting 90% PIs were overlaid over the observed PK levels in such populations (Hornik, 2017). Moreover, the percentage of observed mean concentrations () that fall outside the 5th and 95th quantiles of the PI of the PopPBPK model simulations were estimated (Maharaj, 2019): …”

The development of a population physiologically based pharmacokinetic model for mycophenolic mofetil and mycophenolic acid in humans using data from plasma, saliva, and kidney tissue

“…Visual predictive checks were done by overlaying a 90% predictive interval (PI) of Pop‐PBPK model‐simulated MPA, MPAG, and MMF levels in plasma, saliva, and kidney tissues at each time point over previously reported respective observed PK data (Hornik et al, ). Numerical predictive checks were achieved by calculating the percentage of observed data points ( ) representing the mean reported measured concentration that falls outside the 5th ( ) and 95th ( ) quantiles of the 90% PI of the Pop‐PBPK simulations ( or ) model using the following formula (Maharaj, Wu, Hornik, & Cohen‐Wolkowiez, ): …”