A Primer on Maximum Causal Entropy Inverse Reinforcement Learning

Gleave, Adam; Toyer, Sam

doi:10.48550/arxiv.2203.11409

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the original feature matching objective only considers reward functions that are linear in φ, state-of-the-art methods are scalable to complex problems by parameterizing R with a deep neural network (Wulfmeier et al, 2015;Ho & Ermon, 2016). For an elaborate introduction to MCE-IRL we refer the reader to the work of Gleave & Toyer (2022).…”

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

Inverse reinforcement learning through logic constraint inference

2023

View full text Add to dashboard Cite

Autonomous robots start to be integrated in human environments where explicit and implicit social norms guide the behavior of all agents. To assure safety and predictability, these artificial agents should act in accordance with the applicable social norms. However, it is not straightforward to define these rules and incorporate them in an agent's policy. Particularly because social norms are often implicit and environment specific. In this paper, we propose a novel iterative approach to extract a set of rules from observed human trajectories. This hybrid method combines the strengths of inverse reinforcement learning and inductive logic programming. We experimentally show how our method successfully induces a compact logic program which represents the behavioral constraints applicable in a Tower of Hanoi and a traffic simulator environment. The induced program is adopted as prior knowledge by a model-free reinforcement learning agent to speed up training and prevent any social norm violation during exploration and deployment. Moreover, expressing norms as a logic program provides improved interpretability, which is an important pillar in the design of safe artificial agents, as well as transferability to similar environments.

show abstract

Section: Inverse Reinforcement Learningmentioning

confidence: 99%