Adaptive independence samplers

Keith, Jonathan M.; Kroese, Dirk P.; Sofronov, Georgy

doi:10.1007/s11222-008-9070-2

Cited by 27 publications

(22 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We adopt the so-called cross-entropy adaptive independence sampler introduced in Keith, Kroese, and Sofronov (2008). Speci…cally, the proposal density is chosen such that the Kullback-Leibler divergence, or the cross-entropy (CE)distance between the proposal density and the target (the posterior density) is minimal, where the CE distance between the densities g 1 and g 2 is de…ned as:…”

Section: Collapsed Sampling With the Cross-entropy Methodsmentioning

confidence: 99%

Estimation in Non-Linear Non-Gaussian State Space Models with Precision-Based Methods

Chan

Strachan

2012

SSRN Journal

View full text Add to dashboard Cite

In recent years state space models, particularly the linear Gaussian version, have become the standard framework for analyzing macroeconomic and …nancial data. However, many theoretically motivated models imply non-linear or non-Gaussian speci…cations -or both. Existing methods for estimating such models are computationally intensive, and often cannot be applied to models with more than a few states. Building upon recent developments in precision-based algorithms, we propose a general approach to estimating high-dimensional non-linear non-Gaussian state space models. The baseline algorithm approximates the conditional distribution of the states by a multivariate Gaussian or t density, which is then used for posterior simulation. We further develop this baseline algorithm to construct more sophisticated samplers with attractive properties: one based on the acceptreject Metropolis-Hastings (ARMH) algorithm, and another adaptive collapsed sampler inspired by the cross-entropy method. To illustrate the proposed approach, we investigate the e¤ect of the zero lower bound of interest rate on monetary transmission mechanism.

show abstract

Section: Collapsed Sampling With the Cross-entropy Methodsmentioning

confidence: 99%

Estimation in Non-Linear Non-Gaussian State Space Models with Precision-Based Methods

Chan

Strachan

2012

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…Other mixture from the exponential family can also be considered, such as a discrete/continuous mixture of Student's t-distribution (see Andrieu and Thoms (2008) for details). Andrieu and Moulines (2006) suggest to fit these parameters using a maximum likelihood approach or, equivalently, by maximizing the cross-entropy between the proposal distribution and the target; this algorithm shares some similarities with the so-called adaptive independence sampler developed in Keith et al (2008). In this framework, the parameters are fitted using a sequential version of the EM algorithm (Cappe and Figure 1.5: Adaptive fit of a mixture of three Gaussian distributions with arbitrary means and covariance using the maximum likelihood approach developed in Andrieu and Moulines (2006) Moulines, 2009) (several improvements on this basic schemes are presented in Andrieu and Thoms (2008)).…”

Section: Internal Adaptive Mcmcmentioning

confidence: 99%

Adaptive Markov chain Monte Carlo: theory and methods

Atchadé

Fort

Moulines

et al. 2011

Bayesian Time Series Models

View full text Add to dashboard Cite

“…Keith et al (2008) developed adaptive independence samplers by minimizing the Kullback-Leibler (KL) divergence in order to provide the best candidate density, which consists of a mixture of Gaussian densities. The minimization of the KL-divergence is done by applying the EM algorithm of Dempster et al (1977) and the number of mixture components is selected through information criteria like AIC (Akaike (1974)), BIC (Schwarz (1978)) or DIC (Gelman et al (2003)).…”

Section: Introductionmentioning

confidence: 99%

A Class of Adaptive EM-Based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation

2011

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in AbstractA class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution -typically a posterior distribution, of which we only require a kernel -in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization(MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis-Hastings (MH) method.The MitISEM algorithm performs well in exploring non-elliptical shapes of posterior and predictive distributions, in estimating predictive likelihoods and forecasting Value at Risk, for several examples of statistical and econometric models. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged.This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach, for importance sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model's mixture regimes' parameters. Third, we propose a partial MitISEM approach, which aims at approximating the marginal and conditional posterior distributions of subsets of model parameters, rather than the joint. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like mixtu...

show abstract

Adaptive independence samplers

Cited by 27 publications

References 32 publications

Estimation in Non-Linear Non-Gaussian State Space Models with Precision-Based Methods

Estimation in Non-Linear Non-Gaussian State Space Models with Precision-Based Methods

Adaptive Markov chain Monte Carlo: theory and methods

A Class of Adaptive EM-Based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation

Contact Info

Product

Resources

About