Divide and Conquer: Recursive Likelihood Function Integration for Hidden Markov Models with Continuous Latent Variables

Abstract

This paper develops a method to efficiently estimate hidden Markov models with continuous latent variables using maximum likelihood estimation. To evaluate the (marginal) likelihood function, I decompose the integral over the unobserved state variables into a series of lower dimensional integrals, and recursively approximate them using numerical quadrature and interpolation. I show that this procedure has very favorable numerical properties: First, the computational complexity grows linearly in the number of periods, making the integration over hundreds and thousands of periods feasible. Second, I prove that the numerical error accumulates sublinearly in the number of time periods integrated, so the total error can be well controlled for a very large number of periods using, for example, Gaussian quadrature and Chebyshev polynomials. I apply this method to the bus engine replacement model of Rust [Econometrica 55(5): 999-1033] to verify the accuracy and speed of the procedure in both actual and simulated data sets.

Adaptive grids for the estimation of dynamic models

“…In particular, this algorithm is formally suitable for the application of any iterative refinement scheme for the grid over state variables of the dynamic problem. Algorithm 2 conceptually extends NFXP with iterative grid updating (for every parameter value), as proposed by Reich (2018). However, if the refinement step in line 6 of Algorithm 2 is discrete (as it is in grid adaption by node insertion), the likelihood function L, which itself depends on the approximation of the value function-and thus on its grid, will generally become discontinuous, as the change of the underlying value function grid is discontinuous itself.…”

“…Consequently, a popular approach to grid creation is iterative refinement: given the grid from the last iteration (or starting from a uniform grid), the unknown function is approximated, and-based on some approximation error criterion-a new grid is created by inserting additional nodes in regions of high approximation error; this procedure is repeated until the maximum approximation error is below some threshold. Iterative grid refinement methods have successfully been applied to dynamic programming problems with continuous state variables in economics; see, for example, Grüne and Semmler (2004), Brumm and Scheidegger (2017), and Reich (2018).…”

“…its flexible counterpart results in roughly 1.5 times longer computation times. The case of the model with a serially correlated, unobserved utility component requires us (i) to approximate the expected value as a function of a two-dimensional state variable, and (ii) to integrate out the random shock when computing the likelihood function; we do the latter by applying the recursive likelihood integration method (RLI; Reich, 2018;Lanz, 2021). Our findings regarding the efficiency of the balanced error approach compared to fixed and uniform grids in the two-dimensional case are comparable, but given the longer absolute computing times, the absolute gains are even more significant.…”

An Instrumental Variable Approach to Dynamic Models

“…Norets (2009) proposes a Bayesian estimation method for dynamic discrete choice models with serially correlated unobservables. Additional full-solution approaches allowing for serial correlation include Blevins (2016) and Reich (2018). Neither discusses identification formally.…”

Robust decision-making under risk and ambiguity

“…See for exampleChristensen and Connault (2019),Iskhakov et al (2016),Reich (2018), andSu and Judd (2012).…”