Properties of Model-Averaged BMDLs: A Study of Model Averaging in Dichotomous Response Risk Estimation

Abstract

Model averaging (MA) has been proposed as a method of accounting for model uncertainty in benchmark dose (BMD) estimation. The technique has been used to average BMD dose estimates derived from dichotomous dose-response experiments, microbial dose-response experiments, as well as observational epidemiological studies. While MA is a promising tool for the risk assessor, a previous study suggested that the simple strategy of averaging individual models' BMD lower limits did not yield interval estimators that met nominal coverage levels in certain situations, and this performance was very sensitive to the underlying model space chosen. We present a different, more computationally intensive, approach in which the BMD is estimated using the average dose-response model and the corresponding benchmark dose lower bound (BMDL) is computed by bootstrapping. This method is illustrated with TiO(2) dose-response rat lung cancer data, and then systematically studied through an extensive Monte Carlo simulation. The results of this study suggest that the MA-BMD, estimated using this technique, performs better, in terms of bias and coverage, than the previous MA methodology. Further, the MA-BMDL achieves nominal coverage in most cases, and is superior to picking the "best fitting model" when estimating the benchmark dose. Although these results show utility of MA for benchmark dose risk estimation, they continue to highlight the importance of choosing an adequate model space as well as proper model fit diagnostics.

Model Averaging Software for Dichotomous Dose Response Risk Estimation

“…By combining estimates from multiple models, MA gives researchers and regulators alike a method that accounts for statistical variability as well as model uncertainty. This method has been shown by Wheeler and Bailer (2007) to yield point estimates with minimal bias and confidence limits with nominal coverage properties, while producing estimates that are superior to picking one single model. This is true even if this model describes the data "better" than the other models used.…”

“…In the situations when the two methods do not give identical parameter estimates, a difference in initial estimates along with the existence of local maxima, within the likelihood of the given model being fit, is usually the cause. A small simulation study by Wheeler and Bailer (2007) showed that there was no clear advantage between either the BMDS software and the MADr-BMD optimization strategy. In most all cases, the software converged to the same value, and when the two failed to achieve the same results, the larger likelihood value was found approximately half the time in the US EPA's software, and half the time in the MADr-BMD package.…”

“…The coverage data for the true model fits, using the US EPA software, was obtained from the study of Wheeler and Bailer (2009), where the MA-BMDL coverage was obtained from the subsequent study of Wheeler and Bailer (2007). These studies were preformed using identical simulation conditions.…”

“…a steep dose-response relationship, which lead to questions about the broader applicability of MA in risk assessment. Subsequently Wheeler and Bailer (2007) modified the above procedure to a more computationally intensive methodology. They found this modified MA procedure, which averaged individual dose-response curves instead of the individual BMDs, more effectively described both the dose-response curve as well as the given benchmark dose often yielding estimated BMDs with minimal bias and lower bound estimates that achieved coverage at the nominally specified 100(1 − α)% level.…”

“…Consequently, this paper introduces a software program that implements the "average-model" MA procedure of Wheeler and Bailer (2007) for dichotomous risk assessment and is freely available to the risk analysis community. The paper proceeds by briefly describing this form of model averaging in the context of risk assessment applied to dichotomous dose-response data, and in the process the software is described.…”