Haptic Assistance via Inverse Reinforcement Learning

Scobee, Dexter R. R.; Royo, Vicenç Rúbies; Tomlin, Claire J.; Sastry, S. Shankar

doi:10.1109/smc.2018.00262

Cited by 10 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, although HSC has been sucessfully tested in a wide diversity of tasks, such as: peg-in-hole [10], grasping objects [11], page turning task [12], home-service tasks [13], bolt-spanner task [5], [6] or achieve goals given a trajectory distribution [14], [15] and [16], [17]. It is possible to see that all of these controllers are designed to deal with fixed tasks or explicit information.…”

Section: Motivationmentioning

confidence: 99%

“…Haptic devices have been tested in several implementations, providing haptic sensations to robots [33] and teleoperated systems improving performance as demonstrated in [34], or covering a wide range of applications assisting in task completion [8], [9], helping to generate trajectories [20], [35], or guiding to follow a reference point or trajectory [14], [15], [16], [17].…”

Section: Haptic Devicesmentioning

confidence: 99%

“…In this context, the use of ML techniques in telerobotics has been growing in the last years as shown by the applications with different approaches, such as LfD in SC [3], [38], [10] and HSC [9] applications and IRL, as demonstrated in [17], or by using RL algorithms: in movements assistance [14], path planing in static and dynamic environments [15] and [16], dexterous telemanipulation in virtual environments [12], in semiautonomous control playing video-games or flying a real quadrotor [27].…”

Section: Robot Learning Applications In Teleroboticsmentioning

confidence: 99%

“…(2-19) Q(s s s, a a a; w w w) , (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20) where (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19) denotes the deterministic parameterized actor (policy) and (2-20) denotes the deterministic parameterized critic (action-value function). Then, the deterministic target parameterized action-value function can be defined as:Q (s s s, a a a; w w w…”

Section: Ddpg Functioningmentioning

confidence: 99%

“…Then, the actor can be updated applying the deterministic version of the policy gradient theorem, presented in (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18) for the mini-batch: The samples used to train the actor and critic functions come from the replay buffer R where are stored all the past transitions (s s s, a a a, r, s s s ) sampled from the environment using the exploration policy. In addition, to build the exploration policy, noise sampled from a suitable noise process N is added to the actor policy.…”

Section: Ddpg Functioningmentioning

confidence: 99%

See 4 more Smart Citations

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Fernandez

Caarls

View full text Add to dashboard Cite

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Haptic Devicesmentioning

confidence: 99%

Section: Robot Learning Applications In Teleroboticsmentioning

confidence: 99%

Section: Ddpg Functioningmentioning

confidence: 99%

Section: Ddpg Functioningmentioning

confidence: 99%

See 3 more Smart Citations

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Fernandez

Caarls

View full text Add to dashboard Cite

show abstract

A survey of inverse reinforcement learning

2022

View full text Add to dashboard Cite

Learning from demonstration, or imitation learning, is the process of learning to act in an environment from examples provided by a teacher. Inverse reinforcement learning (IRL) is a specific form of learning from demonstration that attempts to estimate the reward function of a Markov decision process from examples provided by the teacher. The reward function is often considered the most succinct description of a task. In simple applications, the reward function may be known or easily derived from properties of the system and hard coded into the learning process. However, in complex applications, this may not be possible, and it may be easier to learn the reward function by observing the actions of the teacher. This paper provides a comprehensive survey of the literature on IRL. This survey outlines the differences between IRL and two similar methods - apprenticeship learning and inverse optimal control. Further, this survey organizes the IRL literature based on the principal method, describes applications of IRL algorithms, and provides areas of future research.

show abstract

Inverse reinforcement learning through logic constraint inference

2023

View full text Add to dashboard Cite

Autonomous robots start to be integrated in human environments where explicit and implicit social norms guide the behavior of all agents. To assure safety and predictability, these artificial agents should act in accordance with the applicable social norms. However, it is not straightforward to define these rules and incorporate them in an agent's policy. Particularly because social norms are often implicit and environment specific. In this paper, we propose a novel iterative approach to extract a set of rules from observed human trajectories. This hybrid method combines the strengths of inverse reinforcement learning and inductive logic programming. We experimentally show how our method successfully induces a compact logic program which represents the behavioral constraints applicable in a Tower of Hanoi and a traffic simulator environment. The induced program is adopted as prior knowledge by a model-free reinforcement learning agent to speed up training and prevent any social norm violation during exploration and deployment. Moreover, expressing norms as a logic program provides improved interpretability, which is an important pillar in the design of safe artificial agents, as well as transferability to similar environments.

show abstract

Haptic Assistance via Inverse Reinforcement Learning

Cited by 10 publications

References 25 publications

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

A survey of inverse reinforcement learning

Inverse reinforcement learning through logic constraint inference

Contact Info

Product

Resources

About