Recursive robust estimation and control without commitment

Abstract

In a Markov decision problem with hidden state variables, a posterior distribution serves as a state variable and Bayes' law under an approximating model gives its law of motion. A decision maker expresses fear that his model is misspecified by surrounding it with a set of alternatives that are nearby when measured by their expected log likelihood ratios (entropies). Martingales represent alternative models. A decision maker constructs a sequence of robust decision rules by pretending that a sequence of minimizing players choose increments to a martingale and distortions to the prior over the hidden state. A risk sensitivity operator induces robustness to perturbations of the approximating model conditioned on the hidden state. Another risk sensitivity operator induces robustness to the prior distribution over the hidden state. We use these operators to extend the approach of Hansen and Sargent (1995) to problems that contain hidden states.

Learning and Model Validation

“…20 See, e.g., Brock, Durlauf, and West (2007) for an application of Bayesian model averaging to macroeconomic policy. 21 See Sargent (1993), Hansen and Sargent (2007b), Kreps (1998), and Bray and Kreps (1987).…”

“…Finally, perhaps the most interesting and important extension would be to allow the agent to entertain doubts about the entire model class itself. The work of Hansen and Sargent (2007a) on robust filtering of discrete hidden states offers one route toward such an extension. Another possibility is to take advantage of recent work on calibrated learning in repeated games.…”

“…Traditional adaptive control methods are designed to handle coefficient uncertainty, but not model uncertainty. Recently developed robust control methods are designed to handle general forms of model uncertainty, assuming the decision maker starts with an exogenously specified reference model against which all uncertainty can be measured (Hansen and Sargent (2007b)). Bayesian methods could in principle be applied, but would require specification of high dimensional prior distributions.…”

“…Unfortunately, genuine model uncertainty is infinite dimensional, and specifying priors over infinite dimensional objects is not easy. Recent work on robust control opens the door to an analysis of model uncertainty in situations where it may be difficult to specify a prior over models (Hansen and Sargent (2007a)). However, none of these approaches permits feedback from beliefs about models to the actual DGP.…”

Robust Control of Partially Observable Failing Systems

“…In the constraint approach, the set of alternative models is represented as a hard constraint, and confidence in the nominal is captured by the size of this uncertainty set (see, e.g., Ben-Tal and Nemirovski 1998, 1999, 2000Bertsimas and Sim 2004;El Ghaoui and Lebret 1997;Iyengar 2005;Li and Kwon 2013;Nilim and El Ghaoui 2005;Wiesemann et al 2013). The penalty approach on the other hand expresses confidence in the nominal by penalizing alternative models that deviate too far from the nominal, and does so via a penalty function (soft constraint) that appears in the objective function (see, e.g., Dai Pra et al 1996, Peterson et al 2000, Hansen and Sargent 2007, Jain et al 2010, Kim and Lim 2015, Lim and Shanthikumar 2007. In this paper, we adopt an entropy penalty approach to represent model missspecification, because it provides a number of advantages from the perspective of characterizing the structure of the optimal robust control policy.…”