Information Relaxations and Duality in Stochastic Dynamic Programs

Abstract

We describe a general technique for determining upper bounds on maximal values (or lower bounds on minimal costs) in stochastic dynamic programs. In this approach, we relax the nonanticipativity constraints that require decisions to depend only on the information available at the time a decision is made and impose a "penalty" that punishes violations of nonanticipativity. In applications, the hope is that this relaxed version of the problem will be simpler to solve than the original dynamic program. The upper bounds provided by this dual approach complement lower bounds on values that may be found by simulating with heuristic policies. We describe the theory underlying this dual approach and establish weak duality, strong duality, and complementary slackness results that are analogous to the duality results of linear programming. We also study properties of good penalties. Finally, we demonstrate the use of this dual approach in an adaptive inventory control problem with an unknown and changing demand distribution and in valuing options with stochastic volatilities and interest rates. These are complex problems of significant practical interest that are quite difficult to solve to optimality. In these examples, our dual approach requires relatively little additional computation and leads to tight bounds on the optimal values.

Relaxations of Approximate Linear Programs for the Real Option Management of Commodity Storage

“…As in LMS, we use the information relaxation and duality approach for upper bound estimation discussed by Brown et al (2010), which generalizes earlier work by Rogers (2002), Andersen and Broadie (2004), and Haugh and Kogan (2004). However, our approach is more general than the one of LMS.…”

“…For upper bound estimation, we use the information relaxation and duality approach for MDPs (see Brown et al 2010, and references therein). We sample a sequence of spot price and prompt month futures price pairs…”

“…To address this intractability, LMS propose an Approximate Dynamic Program (ADP) based on a value function that in each stage only depends on the spot price, in addition to the inventory level, and ignores all the other elements of the forward curve. Applied to natural gas instances, their model computes near optimal policies, provided it is sequentially reoptimized, and fairly tight dual upper bounds (Glasserman 2004, Chapter 8, andBrown et al 2010). This Storage ADP (SADP) features a peculiar conditional expectation that makes it solvable.…”

Upper bounds on the Bayes-optimal procedure for ranking & selection with independent normal priors

“…Our focus on obtaining upper bounds for sequential decisionmaking problems is also similar in spirit to recent work on information relaxations in (Brown, Smith, and Sun 2010, Brown and Smith 2011, Haugh and Kogan 2004, Rogers 2002.…”

Generalized decision rule approximations for stochastic programming via liftings

“…Similar bounds are often missing in descriptions of ADP methods. A notable exception are the popular performance bounds based on information relaxations [18], which admit an interpretation as restricted dual stochastic programs. We further remark that the decision rule approach can even find near-optimal solutions for stochastic programs without relatively complete recourse [17].…”