A simulation-intensive approach for checking hierarchical models

Dey, Dipak K.; Gelfand, Alan E.; Swartz, Tim B.; Vlachos, Pantelis

doi:10.1007/bf02565116

“…While cross-validation is often considered a gold standard for diagnostics [29,30,31] , it is computationally expensive and may be impractical for very large datasets. Alternatives such as "testing" datasets [32,33] , importance sampling [29] , simulationbased model checking [34] , posterior predictive checks [35,36] , and approaches that balance bias with the computational burden of cross-validation [37,38] may also be used.…”

Section: From Information To Knowledgementioning

confidence: 99%

Environmental Informatics

Cressie¹,

Burden²,

Shumack³

et al. 2017

Wiley StatsRef: Statistics Reference Online

0

View full text Add to dashboard Cite

Environmental Informatics uses a panoply of tools from the Statistics, Mathematics, Computing, and Visualisation disciplines. It uses these tools to reveal, quantify, and validate scientific hypotheses in the environmental sciences, with the quantification of uncertainty central to its approach. There is now a strong recognition that scientific models need to incorporate stochastic components throughout: While it has always been recognised that data have a component of measurement error, attention is now being given to the quantification of model error, and it is becoming accepted by environmental scientists that probability models for the latter allows for a coherent way to make scientific inference. In Environmental Informatics, uncertainty may be assigned not only to datasets of measurements, but also to computer-generated climate-model output. Methodological advances, in the form of hierarchical statistical models and the accompanying computational developments, have expanded the scope of statistical analyses into very large spatial domains. This has led to studies of the dynamical evolution of entire spatial fields of geophysical variables, where results are given in terms of predictive distributions. Environmental Informatics is not only involved in characterising the environment, it can also be used to make decisions about mitigation and adaptation strategies. The steps taken by environmental scientists, from data to information, from information to knowledge, and from knowledge to decisions, are all taken in the presence of uncertainty. Environmental Informatics encompasses all these aspects. Abstract Environmental Informatics uses a panoply of tools from the Statistics, Mathematics, Computing, and Visualisation disciplines. It uses these tools to reveal, quantify, and validate scientific hypotheses in the environmental sciences, with the quantification of uncertainty central to its approach. There is now a strong recognition that scientific models need to incorporate stochastic components throughout: While it has always been recognised that data have a component of measurement error, attention is now being given to the quantification of model error, and it is becoming accepted by environmental scientists that probability models for the latter allows for a coherent way to make scientific inference. In Environmental Informatics, uncertainty may be assigned not only to datasets of measurements, but also to computer-generated climate-model output. Methodological advances, in the form of hierarchical statistical models and the accompanying computational developments, have expanded the scope of statistical analyses into very large spatial domains. This has led to studies of the dynamical evolution of entire spatial fields of geophysical variables, where results are given in terms of predictive distributions. Environmental Informatics is not only involved in characterising the environment, it can also be used to make decisions about mitigation and adaptation strategies. The steps taken by environmental...

show abstract

“…While cross-validation is often considered a gold standard for diagnostics [29,30,31] , it is computationally expensive and may be impractical for very large datasets. Alternatives such as "testing" datasets [32,33] , importance sampling [29] , simulationbased model checking [34] , posterior predictive checks [35,36] , and approaches that balance bias with the computational burden of cross-validation [37,38] may also be used. Inference (hypothesis testing or prediction) is typically needed for answering a number of questions that decision-makers might have.…”

Section: From Information To Knowledgementioning

confidence: 99%

Buried treasures

2014

Past, Present, and Future of Statistical Science

1

0

View full text Add to dashboard Cite

Environmental Informatics uses a panoply of tools from the Statistics, Mathematics, Computing, and Visualisation disciplines. It uses these tools to reveal, quantify, and validate scientific hypotheses in the environmental sciences, with the quantification of uncertainty central to its approach. There is now a strong recognition that scientific models need to incorporate stochastic components throughout: While it has always been recognised that data have a component of measurement error, attention is now being given to the quantification of model error, and it is becoming accepted by environmental scientists that probability models for the latter allows for a coherent way to make scientific inference. In Environmental Informatics, uncertainty may be assigned not only to datasets of measurements, but also to computer-generated climate-model output. Methodological advances, in the form of hierarchical statistical models and the accompanying computational developments, have expanded the scope of statistical analyses into very large spatial domains. This has led to studies of the dynamical evolution of entire spatial fields of geophysical variables, where results are given in terms of predictive distributions. Environmental Informatics is not only involved in characterising the environment, it can also be used to make decisions about mitigation and adaptation strategies. The steps taken by environmental scientists, from data to information, from information to knowledge, and from knowledge to decisions, are all taken in the presence of uncertainty. Environmental Informatics encompasses all these aspects. Abstract Environmental Informatics uses a panoply of tools from the Statistics, Mathematics, Computing, and Visualisation disciplines. It uses these tools to reveal, quantify, and validate scientific hypotheses in the environmental sciences, with the quantification of uncertainty central to its approach. There is now a strong recognition that scientific models need to incorporate stochastic components throughout: While it has always been recognised that data have a component of measurement error, attention is now being given to the quantification of model error, and it is becoming accepted by environmental scientists that probability models for the latter allows for a coherent way to make scientific inference. In Environmental Informatics, uncertainty may be assigned not only to datasets of measurements, but also to computer-generated climate-model output. Methodological advances, in the form of hierarchical statistical models and the accompanying computational developments, have expanded the scope of statistical analyses into very large spatial domains. This has led to studies of the dynamical evolution of entire spatial fields of geophysical variables, where results are given in terms of predictive distributions. Environmental Informatics is not only involved in characterising the environment, it can also be used to make decisions about mitigation and adaptation strategies. The steps taken by environmental...

show abstract

“…The results displayed in Figure Dey et al (1998) proposed a computationally intensive method for checking the suitability of proper priors used in a hierarchical model. Bayarri and Castellanos (2007) In this subsection, we study the performance of the MCMC-IS methods described in subsection 2.2 in estimating the quantiles of the posterior distributions p(d(y (r) , θ)|y (r) ) described in Step 1.…”

Section: The Bayesian Lassomentioning

confidence: 99%

“…The MCMC-IS methods are applied to simulation-based model checking for the heart transplant dataset and model from section 5 of Dey et al (1998). This dataset contains the number of patients developing problems leading to short term organ rejection and the total number of transplant patients at 10 centers and grouped into 5 age groups.…”

Section: Simulation-based Model Checkingmentioning

confidence: 99%

“…One case of particular interest is where it is necessary to evaluate multiple posterior distributions for which the prior is fixed but the datasets forming the posterior distributions are related but substantially different. This situation arises in evaluating certain Bayesian p-values such as those discussed in Dey et al (1998); Bayarri and Berger (2000); Robins et al (2000); Bayarri and Castellanos (2007); Fraser and Rousseau (2008) and in simulation studies evaluating the performance of a Bayesian estimator Gadja et al (2010). In this situation, it is not clear that the original posterior distribution will provide an adequate importance function.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recentered Importance Sampling With Applications to Bayesian Model Validation

McVinish

¹

,

Mengersen

²

,

Nur

³

et al. 2012

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

Since its introduction in the early 90's, the idea of using importance sampling (IS) with Markov chain Monte Carlo (MCMC) has found many applications. This paper examines problems associated with its application to repeated evaluation of related posterior distributions with a particular focus on Bayesian model validation. We demonstrate that, in certain applications, the curse of dimensionality can be reduced by a simple modification of IS. In addition to providing new theoretical insight into the behaviour of the IS approximation in a wide class of models, our result facilitates the implementation of computationally intensive Bayesian model checks. We illustrate the simplicity, computational savings and potential inferential advantages of the proposed approach through two substantive case studies, notably computation of Bayesian p-values for linear regression models and simulation-based model checking. Supplementary materials including appendices and the R code for Section 3.1.2 are available online.

show abstract

A simulation-intensive approach for checking hierarchical models

Abstract: Discrepancy measures, generalized linear mixed model, Monte Carlo tests, sampling-based model fitting, stagewise model adequacy, 62A15, 62E25, 62-07,

Cited by 56 publications

References 16 publications

Environmental Informatics

Environmental Informatics

Buried treasures

Recentered Importance Sampling With Applications to Bayesian Model Validation

Contact Info

Product

Resources

About