Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Fernandez, Franklin Cardenoso; Caarls, Wouter

doi:10.17771/pucrio.acad.50379

DOI: 10.17771/pucrio.acad.50379

|View full text |Cite

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Franklin Cardenoso Fernandez

Wouter Caarls

Abstract: Cardeñoso Fernandez, Franklin; Caarls, Wouter (Advisor). Deep reinforcement learning for haptic shared control in unknown tasks. Rio de Janeiro, 2020. 123p.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(1 citation statement)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This restricts the optimality because a large domain of states and actions were left unexplored. Fernandez et al used DDPG for the haptic shared control in unknown tasks [15]. Li et al used the actor-critic RL approach for human-robot shared control in robotic arms [16].…”

mentioning

confidence: 99%

Deep reinforcement learning for shared control of mobile robots

Tian

Shaik

Wang

2021

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

Shared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance. However, prior work that seeks to find the optimal shared control ratio needs an accurate human model, which is usually challenging to obtain. In this study, the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient (DDPG) (TD3X)-based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally. The robot's states, shared control ratio in the previous time step, and human's control input is used as inputs to the reinforcement learning (RL) agent, which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model. Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio. Furthermore, to accelerate the training process and protect the robot, a navigation demonstration policy and a safety guard are developed. A neural network (NN) structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed. In addition, an extended DAGGER (DAGGERX) human agent is developed for training the RL agent to reduce human workload. Robot simulations and experiments with humans in the loop are conducted. The results show that the DAGGERX human agent can simulate real human inputs in the worst-case scenarios with a mean square error of 0.0039. Compared to the original TD3 agent, the TD3X-based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment. The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment, while the performance is equally good in the complex environment because of the use of an advanced human agent. In the human subject tests, participants' average perceived workload was significantly lower in shared control than that in exclusively manual control (26.90 vs. 40.07, p = 0.013).

show abstract

mentioning

confidence: 99%

Deep reinforcement learning for shared control of mobile robots

Tian

Shaik

Wang

2021

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Abstract: Cardeñoso Fernandez, Franklin; Caarls, Wouter (Advisor). Deep reinforcement learning for haptic shared control in unknown tasks. Rio de Janeiro, 2020. 123p.

Cited by 1 publication

References 43 publications

Deep reinforcement learning for shared control of mobile robots

Deep reinforcement learning for shared control of mobile robots

Contact Info

Product

Resources

About