Efficient Hybrid-Supervised Deep Reinforcement Learning for Person Following Robot

Pang, Linzhuo; Zhang, Yunzhou; Coleman, Sonya; Cao, He

doi:10.1007/s10846-019-01030-0

Cited by 20 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pang et al [9] applied a hybrid method of supervised learning (SL) and DRL to train an agent with deep Q-network (DQN) for developing a robot which is able to continuously follow a specified target person from behind. In order to generate the robot's actions, the study uses images obtained from an RGB camera attached to the robot as states which are then fed directly to the neural networks inside the agent.…”

Section: A Drl For Person-following Robotsmentioning

confidence: 99%

“…Instead, the policy is updated through rewards obtained from the interaction of an RL agent with its environment during the training process. Recently, several studies such as [6] and [9] have shown that DRL can be successfully implemented to train agents for obtaining the policy for a robot to generate appropriate actions to follow a specified target person. However, the person-following task to be performed by the robot in [9] is not seen as an integrated substantial task so that another important sub task, namely the obstacle avoidance is neglected.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, several studies such as [6] and [9] have shown that DRL can be successfully implemented to train agents for obtaining the policy for a robot to generate appropriate actions to follow a specified target person. However, the person-following task to be performed by the robot in [9] is not seen as an integrated substantial task so that another important sub task, namely the obstacle avoidance is neglected. On the other hand, since the policy in [6] is only focused on the attending task, the robot often fails to minimize collisions while it is away from the target person.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Integrating Multiple Policies for Person-Following Robot Training Using Deep Reinforcement Learning

Dewa

Miura

2021

IEEE Access

View full text Add to dashboard Cite

Given a training environment which follows Markov decision process for a specific task, a deep reinforcement learning (DRL) agent is able to find possible optimal policies which map states of the environment to appropriate actions by repeatedly trying various actions to maximize training rewards. However, the learned policies cannot be reused directly in the training process for other new tasks resulting wasted precious time and resources. To solve this problem, we propose a DRL-based method for training an agent capable of selecting the appropriate policy for current state of the environment from a set of previously trained optimal policies for a given task which can be decomposed into other sub tasks. We implement our proposed method to a person-following robot task training that can be broken down into three sub tasks, namely: navigation, left attending, and right attending. Using the proposed method, the previously learned optimal navigation policy obtained from our previous work is integrated to attending policies which are trained in this study. We also introduce the use of weight-scheduled action smoothing which is able to stabilize actions generated by the agent in the attending task training. Our experiment results show that the proposed method is able to integrate all sub policies using the action smoothing method even though the navigation and the attending policies have dissimilar input structures, unalike output ranges, and are trained in different ways. Moreover, our proposed method shows better results compared to training from scratch and training using transfer learning strategy. INDEX TERMSDeep reinforcement learning, policies integration, person-following robot, robot training

show abstract

Section: A Drl For Person-following Robotsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Integrating Multiple Policies for Person-Following Robot Training Using Deep Reinforcement Learning

Dewa

Miura

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We use continuous action space [23] so that the agent will generate more dimension output actions resulting more smooth robot's trajectories when it navigates [47]. The continuous action space a ∈ A is defined as…”

Section: ) Action Spacementioning

confidence: 99%

A Framework for DRL Navigation With State Transition Checking and Velocity Increment Scheduling

Dewa

Miura

2020

IEEE Access

View full text Add to dashboard Cite

To train a mobile robot to navigate using end-to-end approach which maps sensors data into actions, we can use deep reinforcement learning (DRL) method by providing training environments with proper reward functions. Although some studies have shown the success of DRL in navigation task for mobile robots, the method needs appropriate hyperparameter settings such as the environment's timestep size and the robot's velocity range to produce a good navigation policy. The previous existing DRL framework has proposed the use of odometry sensor to generate dynamic timestep size in the environment to solve the mismatch problem between the timestep size and the robot's velocity. However, the framework lacks a procedure for checking terminal conditions which may occur during action executions resulting inconsistency in the environment and poor navigation policies. In the case of navigation task, terminal conditions may happen when the robot achieves the navigation goal position or collides with obstacles while performing an action in one timestep. To cope with this problem, we propose a state transition checking method in the DRL environment which is specific for navigation task that leverages odometry and laser sensor to ensure that the environment follows Markov Decision Process with dynamic timestep size. We also introduce a velocity increment scheduling to stabilize the mobile robot during training. Our experiment results show that state transition checking along with the velocity increment scheduling are able to make the robot navigate faster with higher success rate compared to other existing DRL frameworks.

show abstract

“…Thus, the motion trajectory of the robot is not smooth. In addition to the above traditional following methods, [31] proposed an approach in which an agent is trained by hybrid-supervised deep reinforcement learning to perform a person following task in end-to-end manner. However, like some traditional methods, this method does not consider the avoidance of dynamic obstacles, and it's difficult to control the robot to follow people smoothly with action instructions directly generated from discrete images.…”

Section: B Person Followingmentioning

confidence: 99%

A Robust Autonomous Following Method for Mobile Robots in Dynamic Environments

2020

View full text Add to dashboard Cite

Autonomous following is one of the key issues in mobile robotics with a wide range of applications in commercial, industrial and military areas. To achieve autonomous following, the robot needs not only to robustly detect and track the target, but also efficiently follow the target while avoiding obstacles. In this paper, we propose a method that combines a Line-of-sight sensor (a 2D laser range finder) and a Non-lineof-sight sensor (a low-cost Angle-of-Arrival (AOA) tag) to identify, track and follow the target person in dynamic environments. First, in order to get smooth and continuous moving trajectory of the target person, a Kalman Filter is used to fuse person tracking information from AOA data and laser data. Then, a real-time robot-centric rolling grid map is constructed using the laser data. On top of the rolling grid map, a target potential field is generated by using the fast marching method, and then a direction gradient field is created based on the target potential field. With the rolling grid map, the target potential field and the direction gradient field, an improved dynamic window algorithm, FMM-DWA, is proposed to control the robot to move towards the target. The proposed method considers not only the distance between the robot and the target, but also the difference between the heading of the robot and the reference direction provided by the direction gradient field, to avoid falling into the local optimum. To validate the performance of the proposed method, a series of person following experiments are carried out in complex dynamic environments. The experimental results show that the proposed following algorithm can effectively deal with the occlusion problem and robustly follow the target person while quickly avoiding the static and dynamic obstacles.

show abstract

Efficient Hybrid-Supervised Deep Reinforcement Learning for Person Following Robot

Cited by 20 publications

References 28 publications

Integrating Multiple Policies for Person-Following Robot Training Using Deep Reinforcement Learning

Integrating Multiple Policies for Person-Following Robot Training Using Deep Reinforcement Learning

A Framework for DRL Navigation With State Transition Checking and Velocity Increment Scheduling

A Robust Autonomous Following Method for Mobile Robots in Dynamic Environments

Contact Info

Product

Resources

About