Abstract:Mobile robots contributed significantly to the intelligent development of human society, and the motion-planning policy is critical for mobile robots. This paper reviews the methods based on motionplanning policy, especially the ones involving Deep Reinforcement Learning (DRL) in the unstructured environment. The conventional methods of DRL are categorized to value-based, policy-based and actorcritic-based algorithms, and the corresponding theories and applications are surveyed. Furthermore, the recently-emerg… Show more
“…With scenarios containing moving elements, in long-range (global path planning) scenarios, the use of Evolutionary methods is adequate. The latest Artificial Intelligence methods, including the DL and RL methods, still need to be further studied to obtain solid conclusions, as also remarked by Sun et al [ 21 ]. Artificial Intelligence methods based on Fuzzy rules or neural networks can be used for fast Local Planning as an alternative to Reactive Manoeuvre methods.…”
Section: Summary and Conclusionmentioning
confidence: 96%
“…Faust et al [ 158 ] combined RL with the Probabilistic Roadmap Method (PRM), which is one of the algorithms detailed next. For more information about planning algorithms based on RL, refer to the work of Sun et al [ 21 ].…”
Providing mobile robots with autonomous capabilities is advantageous. It allows one to dispense with the intervention of human operators, which may prove beneficial in economic and safety terms. Autonomy requires, in most cases, the use of path planners that enable the robot to deliberate about how to move from its location at one moment to another. Looking for the most appropriate path planning algorithm according to the requirements imposed by users can be challenging, given the overwhelming number of approaches that exist in the literature. Moreover, the past review works analyzed here cover only some of these approaches, missing important ones. For this reason, our paper aims to serve as a starting point for a clear and comprehensive overview of the research to date. It introduces a global classification of path planning algorithms, with a focus on those approaches used along with autonomous ground vehicles, but is also extendable to other robots moving on surfaces, such as autonomous boats. Moreover, the models used to represent the environment, together with the robot mobility and dynamics, are also addressed from the perspective of path planning. Each of the path planning categories presented in the classification is disclosed and analyzed, and a discussion about their applicability is added at the end.
“…With scenarios containing moving elements, in long-range (global path planning) scenarios, the use of Evolutionary methods is adequate. The latest Artificial Intelligence methods, including the DL and RL methods, still need to be further studied to obtain solid conclusions, as also remarked by Sun et al [ 21 ]. Artificial Intelligence methods based on Fuzzy rules or neural networks can be used for fast Local Planning as an alternative to Reactive Manoeuvre methods.…”
Section: Summary and Conclusionmentioning
confidence: 96%
“…Faust et al [ 158 ] combined RL with the Probabilistic Roadmap Method (PRM), which is one of the algorithms detailed next. For more information about planning algorithms based on RL, refer to the work of Sun et al [ 21 ].…”
Providing mobile robots with autonomous capabilities is advantageous. It allows one to dispense with the intervention of human operators, which may prove beneficial in economic and safety terms. Autonomy requires, in most cases, the use of path planners that enable the robot to deliberate about how to move from its location at one moment to another. Looking for the most appropriate path planning algorithm according to the requirements imposed by users can be challenging, given the overwhelming number of approaches that exist in the literature. Moreover, the past review works analyzed here cover only some of these approaches, missing important ones. For this reason, our paper aims to serve as a starting point for a clear and comprehensive overview of the research to date. It introduces a global classification of path planning algorithms, with a focus on those approaches used along with autonomous ground vehicles, but is also extendable to other robots moving on surfaces, such as autonomous boats. Moreover, the models used to represent the environment, together with the robot mobility and dynamics, are also addressed from the perspective of path planning. Each of the path planning categories presented in the classification is disclosed and analyzed, and a discussion about their applicability is added at the end.
“…A jointly learnable behavior and trajectory planner for self-driving vehicles was introduced in [31]. Unlike the majority of neural planning methods that rely on paths demonstrated by experts, we apply a deep reinforcement learning approach [4], thus conserving both the time and human effort in the training phase.…”
Section: Related Workmentioning
confidence: 99%
“…Here machine learning comes to the rescue, as modern methods, like deep neural networks (DNN), make it possible to learn even complicated decision-making policies in constrained state spaces [4]. We have explored this idea in our recent paper [5], where we presented a neural network architecture and a training procedure that allow a local motion planner to learn from its own experience (Fig.…”
This paper demonstrates how an efficient representation of the planned path using B-splines, and a construction procedure that takes advantage of the neural network's inductive bias, speed up both the inference and training of a DNN-based motion planner. We build upon our recent work on learning local car maneuvers from past experience using a DNN architecture, introducing a novel B-spline path construction method, making it possible to generate local maneuvers in almost constant time of about 11 ms, respecting a number of constraints imposed by the environment map and the kinematics of a car-like vehicle. We evaluate thoroughly the new planner employing the recent Bench-MR framework to obtain quantitative results showing that our method outperforms stateof-the-art planners by a large margin in the considered task.
“…The above-mentioned methods improved the disadvantages of DRL in unstructured environments, enhancing the performance of the models and improving training efficiency. The DRL-based tasks were performed by calculating cumulative rewards to obtain an optimal policy model, which would have a better performance when a large amount of high-value training samples were available [26]. However, for the fruit-picking task, there were too few valid samples at the beginning of the training due to the randomness and uncertainty of the target fruit and obstacle locations.…”
When using deep reinforcement learning algorithms for path planning of a multi-DOF fruit-picking manipulator in unstructured environments, it is much too difficult for the multi-DOF manipulator to obtain high-value samples at the beginning of training, resulting in low learning and convergence efficiency. Aiming to reduce the inefficient exploration in unstructured environments, a reinforcement learning strategy combining expert experience guidance was first proposed in this paper. The ratios of expert experience to newly generated samples and the frequency of return visits to expert experience were studied by the simulation experiments. Some conclusions were that the ratio of expert experience, which declined from 0.45 to 0.35, was more effective in improving learning efficiency of the model than the constant ratio. Compared to an expert experience ratio of 0.35, the success rate increased by 1.26%, and compared to an expert experience ratio of 0.45, the success rate increased by 20.37%. The highest success rate was achieved when the frequency of return visits was 15 in 50 episodes, an improvement of 31.77%. The results showed that the proposed method can effectively improve the model performance and enhance the learning efficiency at the beginning of training in unstructured environments. This training method has implications for the training process of reinforcement learning in other domains.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.