Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM

Park, Kwan-Woo; Kim, MyeongSeop; Kim, Jung-Su; Park, Jae‐Han

doi:10.3390/app12199837

Cited by 10 publications

(9 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the algorithm still faced the problem of sparse environmental rewards. Park, K.-W. et al [ 32 ] employed the SAC algorithm to solve the path planning problem for multi-arm manipulators to avoid fixed and moving obstacles, and used the LSTM network to predict the position of the moving obstacles. The simulation and experimental results showed the optimal path and good prediction of obstacle position.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm

Zhang,

Chen

2023

Sensors

View full text Add to dashboard Cite

This paper proposes an improved Soft Actor–Critic Long Short-Term Memory (SAC-LSTM) algorithm for fast path planning of mobile robots in dynamic environments. To achieve continuous motion and better decision making by incorporating historical and current states, a long short-term memory network (LSTM) with memory was integrated into the SAC algorithm. To mitigate the memory depreciation issue caused by resetting the LSTM’s hidden states to zero during training, a burn-in training method was adopted to boost the performance. Moreover, a prioritized experience replay mechanism was implemented to enhance sampling efficiency and speed up convergence. Based on the SAC-LSTM framework, a motion model for the Turtlebot3 mobile robot was established by designing the state space, action space, reward function, and overall planning process. Three simulation experiments were conducted in obstacle-free, static obstacle, and dynamic obstacle environments using the ROS platform and Gazebo9 software. The results were compared with the SAC algorithm. In all scenarios, the SAC-LSTM algorithm demonstrated a faster convergence rate and a higher path planning success rate, registering a significant 10.5 percentage point improvement in the success rate of reaching the target point in the dynamic obstacle environment. Additionally, the time taken for path planning was shorter, and the planned paths were more concise.

show abstract

Section: Related Workmentioning

confidence: 99%

“…When it comes to the SAC-LSTM algorithm, the inclusion of LSTM layers introduces additional time-dependency computations. A smaller batch size (32) might reduce the computational load per training iteration, but the burn-in process and LSTM's time dependencies increase the computational load batch.…”

Section: Computational Load Of Neural Networkmentioning

confidence: 99%

Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm

Zhang,

Chen

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Estas abordagens foram: aprendizado de movimentos ponto a ponto [3], aprendizado com feedback interativo [27], algoritmos de interac ¸ão contínua [24], manipulac ¸ão de objetos [35], tarefa de abertura de porta [45], manipulac ¸ão coordenada de multi-robôs [20], controle neural adaptativo [42], controle de manipuladores [18], planejamento de trajetória [47], inspec ¸ão robótica [14] e controle de posic ¸ão [49]. [48,17,19,38] Não utilizou [3,27,24,35,45,20,42,18,47,14,49] Verifica-se também que a variedade de manipuladores robóticos utilizados é ampla, sendo o modelo UR3 da Universal Robots o mais utilizado entre estes em trabalhos com enfoque em: Tarefa peg-in-hole [6,4] e controle de brac ¸o duplo robótico [23]; seguido do PANDA [10,34], UR5 [31,22], RM-X52 [32,33] e IRB 1600 [1,2]. Além disso, 3 trabalhos fizeram o uso de manipuladores produzidos em laboratório, customizados ou com pec ¸as impressas em 3D, implementados em: Controle de articulac ¸ões robóticas [36], planejamento de movimento [46] e mapeamento de controlador de brac ¸o robótico [37].…”

Section: A Manipuladores Robóticos E Simuladoresunclassified

Aplicações de Aprendizado por Reforço em Manipuladores Robóticos: Uma Revisão Sistemática

Brito,

Carvalho Ottoni,

Ottoni

2023

Anais Do XVI Congresso Brasileiro De Inteligência Computacional

View full text Add to dashboard Cite

O Aprendizado por Reforço (AR) tem diversas aplicações em tecnologia como um método expoente de resolução de problemas otimizando ações por meio de recompensas. Existe uma gama de estudos aplicados à robótica com o objetivo de aprimorar o estado da arte nesta área. Assim, este trabalho visa discutir as aplicações de AR em manipuladores robóticos através de uma revisão sistemática da literatura, onde foram analisados 38 trabalhos da área publicados entre 2013 e 2023. Desta forma, foram elaboradas 6 perguntas de pesquisa para o desenvolvimento do trabalho. Baseado nestas perguntas, foi possível destacar os resultados da revisão sistemática. Entre as técnicas discutidas, Q-Learning (23,68%), Deep Reinforcement Learning (28,95%), Actor-Critic (26,32%) e Policy Gradient (21,05%) foram as principais. Entre os ambientes e equipamentos para experimentação física e simulada, os mais utilizados foram o simulador Matlab (18,42%) e o manipulador UR3 (7,89%). também foram apresentados resultados sobre o ajuste de hiperparâmetros, sendo que apenas 10,53% dos trabalhos realizaram o ajuste. Além disso, foi realizada uma comparação com outros trabalhos de revisão sistemática do tema proposto. Por fim, foram discutidas as perguntas de pesquisa deste trabalho e apresentadas as principais indicações de trabalhos futuros para promover a continuidade do desenvolvimento de aplicações na área.

show abstract

“…Most current collision detection in simulation environments uses a regular-shaped bounding box to envelop the manipulator as the colliders. The authors in [ 18 ] used a sphere bounding box envelope manipulator for collision detection in path planning, while those of [ 19 , 20 ] used Oriented Bounding Box (OBB) in their studies. Since collision-detection algorithms are simpler for regular-shaped colliders, these regular-shaped bounding boxes can optimize the modeling and computing speed by simplifying the collider structure and improve the detection efficiency at the cost of a loss of the detection accuracy.…”

Section: Introductionmentioning

confidence: 99%

A Self-Collision Detection Algorithm of a Dual-Manipulator System Based on GJK and Deep Learning

Zhi

Adili

et al. 2023

Sensors

View full text Add to dashboard Cite

Self-collision detection is fundamental to the safe operation of multi-manipulator systems, especially when cooperating in highly dynamic working environments. Existing methods still face the problem that detection efficiency and accuracy cannot be achieved at the same time. In this paper, we introduce artificial intelligence technology into the control system. Based on the Gilbert-Johnson-Keerthi (GJK) algorithm, we generated a dataset and trained a deep neural network (DLNet) to improve the detection efficiency. By combining DLNet and the GJK algorithm, we propose a two-level self-collision detection algorithm (DLGJK algorithm) to solve real-time self-collision detection problems in a dual-manipulator system with fast-continuous and high-precision properties. First, the proposed algorithm uses DLNet to determine whether the current working state of the system has a risk of self-collision; since most of the working states in a system workspace do not have a self-collision risk, DLNet can effectively reduce the number of unnecessary detections and improve the detection efficiency. Then, for the working states with a risk of self-collision, we modeled precise colliders and applied the GJK algorithm for fine self-collision detection, which achieved detection accuracy. The experimental results showed that compared to that with the global use of the GJK algorithm for self-collision detection, the DLGJK algorithm can reduce the time expectation of a single detection in a system workspace by 97.7%. In the path planning of the manipulators, it could effectively reduce the number of unnecessary detections, improve the detection efficiency, and reduce system overhead. The proposed algorithm also has good scalability for a multi-manipulator system that can be split into dual-manipulator systems.

show abstract

Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM

Cited by 10 publications

References 28 publications

Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm

Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm

Aplicações de Aprendizado por Reforço em Manipuladores Robóticos: Uma Revisão Sistemática

A Self-Collision Detection Algorithm of a Dual-Manipulator System Based on GJK and Deep Learning

Contact Info

Product

Resources

About