A robust Markov game controller for nonlinear systems

Synergizing reinforcement learning and game theory—A new direction for control

“…A trial is terminated when it successfully balances the pole for 3000 time steps corresponding to 5 min of continuous balancing in real time or till failure. We reproduce results from [31]; Figs. 5 and 6 show performance of Markov game controller in handling the external disturbances and parameter variations.…”

“…Generalization to tackle the 'Curse of Dimensionality' can be introduced using either the neural networks [31,51] or the FIS [52,53]. While one cannot provide strong a-priori guarantees on approximation quality/performance with function approximation in most cases; viability of function approximation for MDP's has been carefully analyzed by Bertsekas and Tsitsiklis [5].…”

“…For detailed results involving Markov game control of two-link robot arm, simulation models, simulation and other parameters, and further results on pendulum swing-up, reader is referred to [31].…”

“…Mathematical models of systems are typically incomplete and/or imprecise and real systems operate in presence of a variety of external disturbances. Proposed Markov game formulation offers an effective platform for designing high performance control systems that can operate in presence of significant and large uncertainty in system models and external disturbances [31].…”

“…Markov game formalism was extended to continuous state-action space problems by proposing a continuous action variant of Minimax-Q. Continuous action neural Markov game control [31] has been found to outperform other game based and nongame based contemporary control schemes, i.e., RL based robust controller [62], and an H ∞ theory based robust game controller [50]. For the inverted pendulum swing-up, each trial is started from an initial state x 0 = ( 0 , 0 ) where  0 and 0 are selected from a uniform distribution that ranges between 0 and 0.01.…”