We propose a partitioning algorithm to solve a class of linear-quadratic Markov decision processes with inequality constraints and nonconvex stagewise cost; within each region of the partitioned state space, the value function and the optimal policy have analytical quadratic and linear forms, respectively. Compared to grid-based numerical schemes, the partitioning algorithm gives the closed-form solution without discretization error, and in many cases does not suffer from the curse of dimensionality. The algorithm is applied to two applications. In the main application, we present a model for limit order books with stochastic market depth to study the optimal order execution problem; stochastic market depth is consistent with empirical studies and necessary to accommodate various order activities. The optimal execution policy obtained by the algorithm significantly outperforms that of a deterministic market depth model in numerical examples. In the second application, we use the algorithm to compute the exact optimal solution to the renewable electricity management problem, for which previously only an approximate solution was known. As a comparison, we show that the approximate solution can be quite inaccurate for some initial states and thus demonstrate an advantage of the exact solution.History: Accepted by Yinyu-Ye, optimization.2 Management Science, Articles in Advance, pp. 1-20, © 2017 INFORMS our numerical examples. For the second application of renewable electricity management in intraday markets, the partitioning algorithm can analytically solve the discrete-time version of the optimal power trading/production problem in Aïd et al. (2016), which has, to our knowledge, previously been solved only approximately. It is numerically demonstrated that for some initial states the approximate solution can perform rather poorly.
Background on Market MicrostructureMarket microstructure concerns how a transaction occurs and its impact on future prices. For example, in LOBs, the execution price essentially depends on the market depth of the LOB. In the model of renewable electricity management, trading activities in intraday markets lead to price impacts that may affect future transactions. Next, we briefly mention the backgrounds of the two applications.An LOB is the list of limit orders that a trading venue uses to record the interest of buyers and sellers. Each limit order specifies a quantity, the intention to buy or sell, and a reservation price. For example, a limit buy order may be placed to purchase 10 shares at $20. Clearly, no limit sell order sets an ask price lower than the bid price of any limit buy order; otherwise, a transaction would occur. Hence, the highest bid price and the lowest ask price give rise to the so-called bid-ask spread. All limit orders in the LOB constitute a profile that reflects demand/supply at each price. Thus, the LOB is characterized by its market depth, i.e., the density of limit order shares at each price. In addition to limit orders, there are market orders, which specify...