Instantaneous Control of Brownian Motion

Abstract

A controller continuously monitors a storage system, such as an Inventory or bank account, whose content Z = {Zt, t > 01 fluctuates 2 as a (p,a ) Brownian motion in the absence of control. Holding costs are incurred continuously at rate h(Zt). At any time, the controller may instantaneously increase the content of the system, incurring a proporitional cost of r times the size of the increase, or decrease the content at a cost of I times the size of the decrease. We consider the case where h is convex on a finite interval [a,R] and h = -outside this interval. The objective is to minimize the expected discounted sum of holding costs and control costs over an Infinite planning horizon.It is shown that there exists an optimal control limit policy, characterized by two parameters a and b (a < a < b < P). Roughly speaking, this policy exerts the minimum amounts of control sufficient to keep Zt c [a,b] for all t > 0. Put another way, the optimal control limit policy imposes on Z a lower reflecting barrier at a and an upper reflecting barrier at b. We do not give a full-blown El algorithm for construction of the optimal control limits, but a computational scheme could easily be developed from our constructive proof of existence. Z t = Xt+R t-t t > 0,and we define the associated cost function . t' k(x) =E ff e-t h(Z)dt +r e-Y dR~ t i eYt dL 0 0with the Riemann-Stieltjes integrals on the right defined to include the control costs rR 0 and AL 0 incurred at t = 0 (see 13). Our objective is to find a policy which minimizes k(x) for every starting state x.An essential feature of this problem is that the controller can instantaneously change the content (or state) of the storage system.Thus, it is possible to further impose state constraints on the controller's actions, which may be formally expressed by setting h(x) = -for some states x. In the same way, one of the controller's options may be eliminated by setting r = -or IThe special case where r <, A < -and In this paper we consider the Instantaneous control problem with r < -, A < and a holding cost function of the formwhere --< a < 0 < -. It will be shown that there exists an optimal control limit policy with lower limit a and upper limit b, where a < a < b < P. We do not present a full-blown algorithm for computation of the optimal control limits a and b, but a computational scheme could easily be developed from our constructive proof of existence. Our treatment generalizes the result by Harrison and Taylor(6] described earlier, and the methods used here are also more elegant and more general in their applicability. This improvement in methodology and presentation has itself been a major goal In our study, although the extension to general convex holding costs Is potentially important for applications.It will ultimately be found that the minimal cost function f for our instantaneous control problem satsifies the optimality Heuristic Derivation of the Optimality EquationTo simplify discussion, we assume In this section that h(x) < for all x eIR (the real line),...

Portfolio Selection with Transaction Costs

“…A natural and rigorous formulation of this problem is a reflected diffusion process, in which the local times of the diffusion at the boundary provide the trading policy. Similar problems have been studied in [1] and [3].…”

A duality approach to continuous-time contracting problems with limited commitment

“…This is a singular control or instantaneous control problem in control theory (e.g., Harrison and Taksar (1983), Fleming and Soner (2006), or Stokey (2008)), where X is the control process and Y and Z are state processes. Note that J andṼ are related by…”

“…The solution X * is related to the classical Skorokhod problem. As is well known (e.g., Harrison and Taksar (1983)), we can express X * as…”

“…Following Harrison and Taksar (1983), we construct an optimal policy as follows. There are two free boundaries satisfying, respectively,…”

“…By contrast, the dual problem transforms the primal problem with participation constraints into an unconstrained problem, which delivers a linear HJB equation subject to free-boundary conditions. Technically, it is an instantaneous (or singular) control problem, similar to those analyzed in Harrison and Taksar (1983), Harrison (1985), and Stokey (2008).…”