The structure (monotonicity, convexity, multimodularity, directional convexity and K‐convexity) of optimal policies is used to reduce the exponential growth with respect to the size of a sequential decision process and to compare different control systems.
Monotonicity can be derived using Tattice programming techniques, sample path analysis or discrete events dynamic programming.
We cover the discrete and continuous‐time MDP and POMDP models with Borel, countable‐and finite‐state and action spaces, as well nonstationary ones.
Applications include admission/control of (networks of) queues, maintenance, inventory, production and innovation.