Markov Decision Processes with Applications to Finance

Bäuerle, Nicole; Rieder, Ulrich

doi:10.1007/978-3-642-18324-9

Cited by 342 publications

(484 citation statements)

References 4 publications

Supporting

Mentioning

465

Contrasting

Unclassified

Order By: Relevance

“…This enables us to focus on finding numerical approximations. To guarantee that these assumptions are satisfied in practical examples, a variety of sufficient conditions have been developed (see [2]). …”

Section: Convex Switching Systemsmentioning

confidence: 99%

“…Optimal stopping problems are an important subclass of Markov decision problems (see Chapters 10 and 11 of [2]), whose upper bound estimation using duality is well studied. As an illustration of our approach, we obtain in section 8.1 bounds on the price of the Bermudan put option, a practically important discrete-time optimal stopping problem.…”

Section: Solution Diagnosticsmentioning

confidence: 99%

“…Consider a realization of the continuous component (z (1) , z (2) ) ∈ R 2 at the current time t = 0, . .…”

Section: T T=0mentioning

confidence: 99%

“…Rigorous mathematical treatments are available in textbook form (see [2], [4], [12], and [23]). However, practical applications remain persistently challenging despite the rich arsenal of theoretical tools that are currently available.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Algorithms for Optimal Control of Stochastic Switching Systems

Hinz

Yap²

2016

Theory Probab. Appl.

View full text Add to dashboard Cite

Abstract. Optimal control problems of switching type with linear state dynamics are ubiquitous in applications of stochastic optimization. For high-dimensional problems of this type, solutions which utilize some convexity related properties are useful. For such problems, we present novel algorithmic solutions which require minimal assumptions while demonstrating remarkable computational efficiency. Furthermore, we devise procedures of the primal-dual kind to assess the distance to optimality of these approximate solutions.Key words. optimal control, American options, pathwise stochastic control, duality DOI. 10.1137/S0040585X97T9879101. Introduction. When making decisions under uncertainty, the major difficulty is to determine how to update estimates and decisions in order to achieve optimality over a given time period. These kinds of questions are often framed within the realm of Markov decision theory, which can be viewed as discrete-time optimal stochastic control.The theoretical underpinnings of Markov decision theory are now well understood. Rigorous mathematical treatments are available in textbook form (see [2], [4], [12], and [23]). However, practical applications remain persistently challenging despite the rich arsenal of theoretical tools that are currently available. In this context, approximate dynamic programming (see [22]) grew from attempts to provide simultaneously practically implementable heuristics and theoretical insights as to why they perform well in practice.In order to control a large system, a practical approach to dealing with the high dimensionality of the state space is to first achieve a finite discretization of it. Alternatively, one can rely on an efficient approximation of functions on this space. In this spirit, function-based methods suggest approximating value functions on the state space. One such method is the least squares Monte Carlo approach, which suggests an approximation by a suitably parameterized set of basis functions. As these parameters are computed by performing successive regressions, this method is placed within the regression-based method family.Following [7], [28], [29], the contribution [18] became the source of subsequent research focused on its theoretical justification. Convergence issues are addressed in [8] and later generalized in [27], [9], and [10], and extensions to multiple exercise rights were considered in [6] and studied in [3], where the connection to statistical learning theory and the theory of empirical processes is emphasized. For an overview of the applications of Monte Carlo methods in financial engineering, we refer the interested reader to Glasserman's book [13] and to the literature cited therein. Beyond financial applications, function approximation methods have also been used to capture local behavior of value functions, and advanced regression methods; e.g., kernel methods [20],

show abstract

Section: Convex Switching Systemsmentioning

confidence: 99%

Section: Solution Diagnosticsmentioning

confidence: 99%

“…Consider a realization of the continuous component (z (1) , z (2) ) ∈ R 2 at the current time t = 0, . .…”

Section: T T=0mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Algorithms for Optimal Control of Stochastic Switching Systems

Hinz

Yap²

2016

Theory Probab. Appl.

View full text Add to dashboard Cite

show abstract

“…With further research and development, decision models became more generalized. Today, Markov decision processes (see [1,8]) are a mathematical framework for making decisions in situations where outcomes are partly random and partly controlled by decision makers.…”

Section: Introductionmentioning

confidence: 99%

Markov decision processes in minimization of expected costs

Rukav

Stražanac²,

Šuvak

et al. 2014

Croat. Oper. Res. Rev.

View full text Add to dashboard Cite

Abstract. Basics of Markov decision processes will be introduced in order to obtain the optimization goal function for minimizing the long-run expected cost. We focus on minimization of such cost of the farmer's policy consisting of different decisions in specific states regarding both milk quality and quantity (lactation states) produced by a dairy cow. The transition probability matrix of the Markov process, used here for modeling of transitions of a dairy cow from one state to another, will be estimated from the data simulated from the lactation model that is often used in practice. We want to choose optimal actions in the states of this Markov process regarding the farmer's costs. This problem can be solved by exhaustive enumeration of all possible cases in order to obtain the optimal policy. However, this is feasible only for a small number of states. Generally, this problem can be approached in the linear programming setting which yields an efficient solution. In order to demonstrate and compare these two approaches, we present an example based on the simulated data regarding milk quality and quantity.

show abstract