York 1982York , 2015 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.Printed on acid-free paper Springer Science+Business Media LLC New York is part of Springer Science+Business Media (www. springer.com) To Juli and Ann I returned, and saw vnder the Sunne, That the race is not to the swift, nor the battell to the strong, neither yet bread to the wise, nor yet riches to men of vnderstanding, nor yet fauour to men of skil; but time and chance happeneth to them all.-Ecclesiastes 9:11 (AV, 1611) Preface to the Second Edition (2015)The theory of probability and stochastic calculus has grown significantly since the publication of the first edition of this book. The theory of stochastic integration and semimartingales, a relatively recent development at the time of the first edition, is now a standard and significant part of the working mathematician's toolkit. Concepts such as Backward SDEs, which were unheard of in 1982 (apart from one paper of Bismut), are now understood to be fundamental to the theory of stochastic control and mathematical finance. Applications of stochastic processes arise particularly in finance and engineering. This book presents a rigorous mathematical framework for these problems in a comprehensive and inclusive way. The general theory of processes was developed in the 1970s by Paul-André Meyer and Claude Dellacherie, but for some years it was little known or appreciated in the 'anglo saxon' world ("sauf que les ingenieurs anglais", as Meyer referred to the remarkable group at Berkeley led by Gene Wong and Pravin Varaiya.). The first edition was an attempt to fill this gap in the English literature.To describe this volume as a second edition is an understatement. The original volume was 300 pages; this has over 650. Consequently, the book contains a large amount of additional material, including much new material.The growth in the discipline over the past 30 years has the consequence that it is even less possible now to attempt to give a comprehen...

By analogy with the theory of Backward Stochastic Differential Equations, we define Backward Stochastic Difference Equations on spaces related to discrete time, finite state processes. This paper considers these processes as constructions in their own right, not as approximations to the continuous case. We establish the existence and uniqueness of solutions under weaker assumptions than are needed in the continuous time setting, and also establish a comparison theorem for these solutions. The conditions of this theorem are shown to approximate those required in the continuous time setting. We also explore the relationship between the driver F and the set of solutions; in particular, we determine under what conditions the driver is uniquely determined by the solution. Applications to the theory of nonlinear expectations are explored, including a representation result.

We present a theory of backward stochastic differential equations in continuous time with an arbitrary filtered probability space. No assumptions are made regarding the left continuity of the filtration, of the predictable quadratic variations of martingales or of the measure integrating the driver. We present conditions for existence and uniqueness of square-integrable solutions, using Lipschitz continuity of the driver. These conditions unite the requirements for existence in continuous and discrete time and allow discrete processes to be embedded with continuous ones. We also present conditions for a comparison theorem and hence construct time consistent nonlinear expectations in these general spaces.Comment: Published in at http://dx.doi.org/10.1214/11-AOP679 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

Most previous contributions to BSDEs, and the related theories of nonlinear expectation and dynamic risk measures, have been in the framework of continuous time diffusions or jump diffusions. Using solutions of BSDEs on spaces related to finite state, continuous time Markov chains, we develop a theory of nonlinear expectations in the spirit of [Dynamically consistent nonlinear evaluations and expectations (2005) Shandong Univ.]. We prove basic properties of these expectations and show their applications to dynamic risk measures on such spaces. In particular, we prove comparison theorems for scalar and vector valued solutions to BSDEs, and discuss arbitrage and risk measures in the scalar case. S. N. COHEN AND R. J. ELLIOTT 3. Basic theorems. Before developing specific applications of these processes, we establish the following results. The methods used are based on those in [7], where similar results are proven for BSDEs on spaces related to Brownian motions.We shall henceforth assume that F is P-a.s. left continuous in t, is Lipschitz continuous as in Theorem 1.1 and satisfiesfor all Y , Z bounded as in Theorem 1.1. Such a driver will be called standard. If also Q ∈ L 2 (F T ), then the pair (F, Q) will be called standard.We shall assume that the rate matrix A of our chain is left continuous, and there is an 0 < ε r < 1 such that it satisfiesdt-a.s., for all i and j, i = j. This assumption is trivially satisfied if the chain X is time-homogenous, and essentially states that we shall not consider chains with positive transition rates unbounded or arbitrarily close to zero.3.1. Various lemmas. Throughout this section, 1 will denote a column vector of appropriate dimension with all components equal to one.We first note that, from the basic properties of stochastic integrals (see, e.g., [14], page 28), the following isometry holds.Lemma 3.1. For any predictable (matrix) process Z (of appropriate dimension) any s < t,Proof. Let ·, · denote the predictable quadratic covariation. Expanding the norms in terms of traces and products, we wish to showthen ]s,t] Z u dM u ]s,t] Z u dM u * − ]s,·] Z u dM u , ]s,·] Z u dM u t BSDES ON MARKOV CHAINS 5 is a uniformly integrable martingale (see [14], page 38), and ]s,·] Z u dM u , ]s,·] Z u dM u t = ]s,t] Z u d M, M u (Z u ) * (by [14], page 48, Theorem 4.4). Combining these equations and taking a trace and an expectation gives the result in this case. If ]s,t] Z u dM u is not square integrable, then, as both sides of (3.2) are nonnegative, they both equal +∞. The result follows. Lemma 3.2. For (F, Q) standard, if Y is the solution to (1.3) given by Theorem 1.1, then Y satisfies sup t∈[0,T ]

We consider backward stochastic differential equations (BSDEs) related to finite state, continuous time Markov chains. We show that appropriate solutions exist for arbitrary terminal conditions, and are unique up to sets of measure zero. We do not require the generating functions to be monotonic, instead using only an appropriate Lipschitz continuity condition.For an F T measurable, R K valued, P-square integrable random variable Q, we shall discuss equations of the formThese functions are assumed to be progressively measurable, i.e. F (., t, Z t , Y t ) and G(., t, Z t ) are F t measurable for all t ∈ [0, T ].We seek a solution of (2), that is a pair (Z, Y ), where Z is an R K valued adapted process and Y is an R K×N adapted process. We shall address this in four stages: firstly we shall show a martingale representation theorem in this framework, then we shall use this to show the existence and uniqueness of solutions in three stages of increasing complexity in F and G. This is essentially the same approach as in [7]; however the details are not the same as our dynamics differ. The key result presented here is Theorem 6.2, an existence and uniqueness result. This result can be seen as a special case of that obtained by [2]; however we here present explicit formulae for certain quantities of which they assume the existence, and, by establishing a martingale representation theorem, do not require the existence of a 'non-hedgeable' process.As a side note, observe that (2) is equivalent to

We consider Backward Stochastic Difference Equations in discrete time with infinitely many states. This paper shows the existence and uniquenss of solutions to these equations in complete generality, and also derives a comparison theorem. Using these, time-consistent nonlinear evaluations and expectations are considered, and it is shown that every such evaluation or expectation corresponds to the solution of a BSDE, without any requirements for continuity or boundedness. The implications of these results in a continuous time context are then considered, and possible applications are discussed.

We consider ergodic backward stochastic differential equations, in a setting where noise is generated by a countable state uniformly ergodic Markov chain. We show that for Lipschitz drivers such that a comparison theorem holds, these equations admit unique solutions. To obtain this result, we show by coupling and splitting techniques that uniform ergodicity estimates of Markov chains are robust to perturbations of the rate matrix, and that these perturbations correspond in a natural way to EBSDEs. We then consider applications of this theory to Markov decision problems with a risk-averse average reward criterion.

In stochastic decision problems, one often wants to estimate the underlying probability measure statistically, and then to use this estimate as a basis for decisions. We shall consider how the uncertainty in this estimation can be explicitly and consistently incorporated in the valuation of decisions, using the theory of nonlinear expectations.