Latent Kullback-Leibler control for dynamic imitation learning of whole-body behaviors in humanoid robots

Ariki, Yuka; Matsubara, Takamitsu; Hyon, Sang-Ho

doi:10.1109/humanoids.2016.7803386

Cited by 4 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The LMDP framework has found applications in robotics (Matsubara et al, 2014;Ariki et al, 2016), crowdsourcing (Abbasi-Yadkori et al, 2015), and controlling the growth dynamics of complex networks (Thalmeier et al, 2017). The related path integral control framework of Kappen (2005) has been applied in several real-world tasks, including robot navigation (Kinjo et al, 2013), motor skill reinforcement learning (Theodorou et al, 2010;Rombokas et al, 2013;, aggressive car maneuvering (Williams et al, 2016) or autonomous flight of teams of quadrotors (Gómez et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

Fast rates for online learning in Linearly Solvable Markov Decision Processes

Neu,

Gómez

2017

Preprint

View full text Add to dashboard Cite

We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost penalizing extreme control inputs. In the current paper, we consider an online setting where the state costs may change arbitrarily between consecutive rounds, and the learner only observes the costs at the end of each respective round. We are interested in constructing algorithms for the learner that guarantee small regret against the best stationary control policy chosen in full knowledge of the cost sequence. Our main result is showing that the smoothness of the control cost enables the simple algorithm of following the leader to achieve a regret of order log 2 T after T rounds, vastly improving on the best known regret bound of order T 3/4 for this setting.

show abstract