A survey of inverse reinforcement learning techniques

Shao, Zhenfeng; Joo, Er Meng

doi:10.1108/17563781211255862

Cited by 74 publications

(48 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, several algorithms were compared empirically [102] and many RL-based techniques currently exist, for example for robotics [50]. A related idea in RL methods is the inverse RL strategy [115], in which the interactive system observes a demonstration and tries to estimate the reward function from that. This way, the agent learns what actually drives the demonstrated behavior (in other words: how can one evaluate the demonstrated behavior as good?…”

Section: Social Learning Strategiesmentioning

confidence: 99%

Machine learning for interactive systems and robots

Cuayáhuitl

Otterlo

Dethlefs

et al. 2013

Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Co

View full text Add to dashboard Cite

Research on interactive systems and robots, i.e. interactive machines that perceive, act and communicate, has applied a multitude of different machine learning frameworks in recent years, many of which are based on a form of reinforcement learning (RL). In this paper, we will provide a brief introduction to the application of machine learning techniques in interactive learning systems. We identify several dimensions along which interactive learning systems can be analyzed. We argue that while many applications of interactive machines seem different at first sight, sufficient commonalities exist in terms of the challenges faced. By identifying these commonalities between (learning) approaches, and by taking interdisciplinary approaches towards the challenges, we anticipate more effective design and development of sophisticated machines that perceive, act and communicate in complex, dynamic and uncertain environments.

show abstract

Section: Social Learning Strategiesmentioning

confidence: 99%

Machine learning for interactive systems and robots

Cuayáhuitl

Otterlo

Dethlefs

et al. 2013

Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Co

View full text Add to dashboard Cite

show abstract

“…We give a brief overview of the standard assumptions existing IRL methods make of the observation data, and mention the main approaches to inference. For a more complete review see, for example, Zhifei and Joo [2012].…”

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

Inverse reinforcement learning from summary data

Kangasrääsiö

Kaski

2018

Mach Learn

View full text Add to dashboard Cite

Inverse reinforcement learning (IRL) aims to explain observed strategic behavior by fitting reinforcement learning models to behavioral data. However, traditional IRL methods are only applicable when the observations are in the form of state-action paths. This assumption may not hold in many real-world modeling settings, where only partial or summarized observations are available. In general, we may assume that there is a summarizing function σ, which acts as a filter between us and the true state-action paths that constitute the demonstration. Some initial approaches to extending IRL to such situations have been presented, but with very specific assumptions about the structure of σ, such as that only certain state observations are missing. This paper instead focuses on the most general case of the problem, where no assumptions are made about the summarizing function, except that it can be evaluated. We demonstrate that inference is still possible. The paper presents exact and approximate inference algorithms that allow full posterior inference, which is particularly important for assessing parameter uncertainty in this challenging inference situation. Empirical scalability is demonstrated to reasonably sized problems, and practical applicability is demonstrated by estimating the posterior for a cognitive science RL model based on an observed user's task completion time only.

show abstract

“…In practice, the reward function can be very hard to specify and exhaustive to tune for sophisticated problems, and this inspires the development of IRL, an extension of RL, which directly tackles this problem by learning the reward function through expert demonstrations [9,10].…”

Section: -4 Inverse Reinforcement Learningmentioning

confidence: 99%