Numerical Methods for Stochastic Control Problems in Continuous Time

Kushner, Harold J.; Dupuis, Paul

doi:10.1007/978-1-4613-0007-6

Cited by 641 publications

(774 citation statements)

References 96 publications

(233 reference statements)

Supporting

Mentioning

769

Contrasting

Unclassified

Order By: Relevance

“…For this approximation one can then apply the Dynamic Programming approach directly, since conditional expectations are transformed into weighted sums. Examples include the (b) Markov Chain approximation method of Kushner and Dupuis (2001) and the (c) Optimal Quantization technique of Bally et al (2005). Implementing optimal policy on a one-dimensional example.…”

Section: Alternative Computational Methodsmentioning

confidence: 99%

Pricing Asset Scheduling Flexibility using Optimal Switching

Carmona

Ludkovski

2008

Applied Mathematical Finance

108

View full text Add to dashboard Cite

We study the financial engineering aspects of operational flexibility of energy assets. The current practice relies on a representation that uses strips of European spark-spread options, ignoring the operational constraints. Instead, we propose a new approach based on a stochastic impulse control framework. The model reduces to a cascade of optimal stopping problems and directly demonstrates that the optimal dispatch policies can be described with the aid of 'switching boundaries', similar to the free boundaries of standard American options. Our main contribution is a new method of numerical solution relying on Monte Carlo regressions. The scheme uses dynamic programming to efficiently approximate the optimal dispatch policy along the simulated paths. Convergence analysis is carried out and results are illustrated with a variety of concrete examples. We benchmark and compare our scheme to alternative numerical methods.

show abstract

Section: Alternative Computational Methodsmentioning

confidence: 99%

Pricing Asset Scheduling Flexibility using Optimal Switching

Carmona

Ludkovski

2008

Applied Mathematical Finance

108

View full text Add to dashboard Cite

show abstract

“…In each region, local policies that drive the system from one region to another are computed, Figure 1. This calculation relies on a samplingbased algorithm called iMDP [23], which approximates the model in (1) using the Markov chain method [18]. Since the probability of transitioning from one region to another depends on the initial state of the stochastic system within the region, a range of probabilities is required to represent transitions between regions.…”

Section: Solutionmentioning

confidence: 99%

“…Let {χ n i , i ∈ Z + } be a controlled Markov chain on M n with probability transition P n and let ∆χ n i = χ i+1 − χ i denote the distance between two consecutive states. In order to maintain the properties of the original system, ∆t n (x) and P n need to satisfy the following local consistency properties [18]:…”

Section: A Discretisation and Local Policiesmentioning

confidence: 99%

Sampling-based stochastic optimal control with metric interval temporal logic specifications

Montana

Dodd

2016

2016 IEEE Conference on Control Applications (CCA)

View full text Add to dashboard Cite

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy.eprints@whiterose.ac.uk https://eprints.whiterose.ac.uk/ Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version -refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher's website. TakedownIf you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Sampling-based Stochastic Optimal Control with Metric Interval Temporal Logic SpecificationsFelipe J. Montana, Jun Liu and Tony J. DoddAbstract-This paper describes a method to find optimal policies for stochastic dynamic systems that maximise the probability of satisfying real-time properties. The method consists of two phases. In the first phase, a coarse abstraction of the original system is created. In each region of the abstraction, a sampling-based algorithm is utilised to compute local policies that allow the system to move between regions. Then, in the second phase, the selection of a policy in each region is obtained by solving a reachability problem on the Cartesian product between the abstraction and a timed automaton representing a real-time specification given as a metric interval temporal logic formula. In contrast to current methods that require a fine abstraction, the proposed method achieves computational tractability by modelling the coarse abstraction of the system as a bounded-parameter Markov decision process (BMDP). Moreover, once the BMDP is created, this can be reused for new specifications assuming the same stochastic system and workspace. The method is demonstrated with an autonomous driving example.

show abstract

“…The number of units x t the investor holds in the asset is given by (3), where x 0 denotes the initial asset position and ξ + t (resp. ξ − t ) represents the cumulative number of units of the asset bought (resp.…”

Section: Utility-based Option Theory In the Presence Of Transaction Cmentioning

confidence: 99%