Neural nets for indirect inference

Abstract

This paper shows how neural networks may be used to approximate the limited information posterior mean, E(θ|Zn), where θ is the parameter vector of a simulable model, and Zn is a vector of statistics. Because the model is simulable, training and testing samples may be generated with sizes large enough to train well a net that is large enough, in terms of number of hidden layers and neurons, to learn E(θ|Zn) with good accuracy. The output of the net can be used as an estimator of the parameter, or, following Jiang et al. (2015), as an input to subsequent classical or Bayesian indirect inference estimation. Targeting E(θ|Zn) using neural nets is simpler, faster, and more successful than is targeting the full information posterior mean, E(θ|Yn), where Yn is the sample data. Code to replicate the examples and to use the methods for other models is available at https://github.com/mcreel/NeuralNetsForIndirectInference.jl. This code uses the Mocha.jl package for the Julia language, which allows for easy access to GPU computing, which greatly accelerates training the net.

Inference Using Simulated Neural Moments

“…The dimension of the statistics used for estimation, Z, can be made minimal (equal to the dimension of the parameter to estimate, θ) by filtering an initial set of statistics, say, W, through a trained neural net. Details of this process are explained in Creel (2017) and references cited therein, and the process is made explicit in the code which accompanies this paper 3 . A summary of this process is: Suppose that W is a p vector of statistics W = W(Y), with p ≥ k, where k = dim θ.…”

“…This paper provides experimental evidence that confidence intervals derived from such estimators may have poor coverage when the moments over-identify the parameters, a result that parallels the above cited results for classical GMM estimators. It goes on to provide evidence that the simulated neural moments that were introduced in Creel (2017), which are just-identifying, when used with MSM-MCMC techniques, cause inferences to become much more reliable, especially when the continuously updating version of the GMM criteria is used. This paper is a continuation of the line of research in Creel (2017), its main new contribution being the experimental confirmation that inferences based upon simulated neural moments are reliable.…”

“…It goes on to provide evidence that the simulated neural moments that were introduced in Creel (2017), which are just-identifying, when used with MSM-MCMC techniques, cause inferences to become much more reliable, especially when the continuously updating version of the GMM criteria is used. This paper is a continuation of the line of research in Creel (2017), its main new contribution being the experimental confirmation that inferences based upon simulated neural moments are reliable. The paper concludes with an example that uses the methods to estimate a jump-diffusion model for returns of the S&P 500 index.…”

“…Experience with this proposal density, as reported below, is that it is easy to tune, by scaling the covariance by a scalar, to achieve an acceptance rate withing the desired limits 4 . Creel (2017) used neural moments to compute a Laplace-type estimator, similarly to what is done here. That paper used nonparametric regression quantiles applied to the set of draws from the Laplace-type posterior to compute confidence intervals, and the posterior draws were generated by a procedure similar to sequential Monte Carlo, rather than MCMC.…”

AKL-ABC: An Automatic Approximate Bayesian Computation Approach Based on Kernel Learning

“…They attempted to use regularization methods for training the neural network but did not obtain significant improvement. Creel [ 31 ] also used DNN to find the posterior mean based on a subset of predefined summary statistics rather than using the full dataset. However, the tuning of a large number of free parameters is still an issue in DNN-based ABC.…”