The Dual Approach to Recursive Optimization: Theory and Examples

Abstract

APPENDIX A: NUMERICAL IMPLEMENTATION THIS SECTION DESCRIBES how to implement the recursive dual approach numerically. Under the conditions of Theorem 2, the dual Bellman operator is a contraction and, consequently, it is natural to calculate D * via value iteration. Numerical approximation of candidate dual value functions is facilitated by their sub-linearity and the simplicity of their domain. The dual Bellman involves an (outer) minimization over a set of multipliers; these multipliers are passed to (and "coordinate") a family of simple (inner) maximizations over current actions and states. Additive separability in the objective may be exploited to decompose the inner maximizations into a family of simpler maximizations that in parametric settings often have analytical solutions.Dual Value Function Approximation. Numerical implementation of a value function iteration algorithm requires approximations to candidate value functions. Our implementation exploits the sub-linearity of dual value functions and uses a piecewise linear approximation (on the spherical domain C). Piecewise linear approximations to value functions defined on spheres were first applied in economics by Judd, Yeltekin, and Conklin (2003). We apply their approximation procedure to our setting. 31 Recall that under the conditions of Theorem 2, the domain for the dual Bellman operator may be identified with an interval of functions D : S × Y → R, each of which is sub-linear in its second argument. As noted, these functions are fully determined on S × C (or a subset thereof). Moreover, their sub-linearity implies that, for all y ∈ C, Judd, Yeltekin, and Conklin (2003) used this approach to approximate the support function of a payoff set in a repeated game; we use it to approximate the recursive dual value function. In other aspects, our (recursive dual) formulation is different from that of Judd, Yeltekin, and Conklin (2003). Alternative approaches to approximation on spherical domains are described in Sloan and Womersley (2000). 32 For this and other properties of sub-linear functions used below, see Florenzano and Van (2001).

Unique Markov Equilibrium Under Limited Commitment

“…Monotone concavity seems a general property when the analysis is relocated in the suitable space of convex feasible sets for continuation values. This provides a method applicable to the study of recursive contracts with incentive constraints, as in Marcet and Marimon (2015), Pavoni et al (2018), and Rustichini (1998).…”

“…Furthermore, a variety of dynamic contracts subject to limited commitment or private information can be studied by recursive methods. The route taken by the prevailing literature privileges a dual method (e.g., Marcet and Marimon (2015) and, more recently, Pavoni et al (2018)). The analysis in this article, instead, shows that a more natural primal (Negishi) approach is feasible for convex recursive programs.…”

“…Rincón-Zapatero and Rodríguez-Palmero (2003) and Martins-da-Rocha and Vailakis (2010) apply the technique to dynamic programming with unbounded returns. Finally, Pavoni et al (2018) exploit monotone concavity of the dual Bellman operator associated with recursive contracts. Differently from previous applications to dynamic programming (Martins-da-Rocha and Vailakis, 2010;Rincón-Zapatero and Rodríguez-Palmero, 2003;Pavoni et al, 2018), in this article concavity of the (Bellman) operator relies on the convexity of the underlaying planning program.…”

“…Finally, Pavoni et al (2018) exploit monotone concavity of the dual Bellman operator associated with recursive contracts. Differently from previous applications to dynamic programming (Martins-da-Rocha and Vailakis, 2010;Rincón-Zapatero and Rodríguez-Palmero, 2003;Pavoni et al, 2018), in this article concavity of the (Bellman) operator relies on the convexity of the underlaying planning program. This property is natural once the (Bellman) operator is innocuously relocated in a sufficiently large domain in which convex combinations of continuation values are allowed.…”

The Demand and Supply of Favours in Dynamic Relationships

“…Marcet and Marimon (2016) use past binding multipliers as history-dependent Pareto weights. Related but distinct is the dual recursive approach developed by Messner, Pavoni, and Sleet (2012) and Pavoni, Sleet, and Messner (2016). Golosov, Tsyvinski, and Werquin (2016) and Ljungqvist and Sargent (2004, Chapters 19 and 20) include comprehensive surveys of the recursive contracts techniques.…”