Reinforcement learning with prior policy guidance for motion planning of dual-arm free-floating space robot

Cao, Yuxue; Wang, Shengjie; Zheng, Xiang; Ma, Wenke; Xie, Xinru; Liu, Lei

doi:10.1016/j.ast.2022.108098

Cited by 11 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Policy Gradients DDPG PPO Other [23,24,39,40,59, (QMIX), [38] (DDQN), [93] (DDQN), [94] (DDQN), [95] (DQN), [96] (DQN) [46][47][48][49][50][97][98][99][100][101][102][103][104][105][106], [22,[52][53][54][55][56][57] [86,88,129-140] (TRPO), [141] (TRPO), [81] (TRPO), [142] (TD3), [143] (SAC), [144] (SAC)…”

Section: Q-networkmentioning

confidence: 99%

“…Unlike the previous works in this domain, the authors used Hindsight Experience Replay (HER) [ 201 ] for sample-efficient training. On the other hand, Cao et al [ 144 ] proposed a DRL framework for a multi-arm manipulator to track trajectories. Similarly to [ 143 ], Cao et al also used SAC as their base algorithm.…”

Section: Multi-robot System Applications Of Multi-agent Deep Reinforc...mentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Orr

Dutta

2023

Sensors

View full text Add to dashboard Cite

Deep reinforcement learning has produced many success stories in recent years. Some example fields in which these successes have taken place include mathematics, games, health care, and robotics. In this paper, we are especially interested in multi-agent deep reinforcement learning, where multiple agents present in the environment not only learn from their own experiences but also from each other and its applications in multi-robot systems. In many real-world scenarios, one robot might not be enough to complete the given task on its own, and, therefore, we might need to deploy multiple robots who work together towards a common global objective of finishing the task. Although multi-agent deep reinforcement learning and its applications in multi-robot systems are of tremendous significance from theoretical and applied standpoints, the latest survey in this domain dates to 2004 albeit for traditional learning applications as deep reinforcement learning was not invented. We classify the reviewed papers in our survey primarily based on their multi-robot applications. Our survey also discusses a few challenges that the current research in this domain faces and provides a potential list of future applications involving multi-robot systems that can benefit from advances in multi-agent deep reinforcement learning.

show abstract

Section: Q-networkmentioning

confidence: 99%

Section: Multi-robot System Applications Of Multi-agent Deep Reinforc...mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Orr

Dutta

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…For single-arm rigid capturing methods, the classical PID control approach is enough to achieve position and velocity control of the end-effector [18]. Nowadays, reinforcement learning (RL)- [19,20], model predictive control (MPC)- [21], and H ∞ -based [22] methods are also researched. Yet, the most crucial problem regarding rigid capturing methods remains the same, which is the difficulty of achieving robust mechanical interaction using rigid structures in space, since rigidness lacks appropriate impact energy dissipation in a frictionless environment.…”

Section: Introductionmentioning

confidence: 99%

Hybrid-Compliant System for Soft Capture of Uncooperative Space Debris

Delisle

Christidi-Loumpasefski

Yalçin

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Active debris removal (ADR) is positioned by space agencies as an in-orbit task of great importance for stabilizing the exponential growth of space debris. Most of the already developed capturing systems are designed for large specific cooperative satellites, which leads to expensive one-to-one solutions. This paper proposed a versatile hybrid-compliant mechanism to target a vast range of small uncooperative space debris in low Earth orbit (LEO), enabling a profitable one-to-many solution. The system is custom-built to fit into a CubeSat. It incorporates active (with linear actuators and impedance controller) and passive (with revolute joints) compliance to dissipate the impact energy, ensure sufficient contact time, and successfully help capture a broader range of space debris. A simulation study was conducted to evaluate and validate the necessity of integrating hybrid compliance into the ADR system. This study found the relationships among the debris mass, the system’s stiffness, and the contact time and provided the required data for tuning the impedance controller (IC) gains. This study also demonstrated the importance of hybrid compliance to guarantee the safe and reliable capture of a broader range of space debris.

show abstract

“…Many researchers have worked on these fields. The United States, Germany, Japan, and other countries have conducted on-orbit space experiments to study the problem of non-cooperative target identification and attitude estimation (Ardaens and Gaias, 2017;D'Amico et al, 2012;Delpech et al, 2012;Noteborn et al, 2011). Many control algorithms of FFSM have also been proposed.…”

Section: Introductionmentioning

confidence: 99%

“…A trajectory planning algorithm of FFSM based on PPO is proposed in Wang et al (2021), which induce the action ensembles method. Actorcritic algorithm is used in Liang et al (2021) for FFSM target capture task, and the control method based on DDPG of a 3degrees-of-freedom (DOF) space manipulator is studied in Du et al (2019). Cao et al (2023 constructed a mixed DRL policy with prior knowledge guidance for FFSM motion planning.…”

Section: Introductionmentioning

confidence: 99%

End-to-end active non-cooperative target tracking of free-floating space manipulators

Lei

2023

Transactions of the Institute of Measurement and Control

View full text Add to dashboard Cite

Active non-cooperative target tracking is an important ability for free-floating space manipulator (FFSM) systems. Aiming at the difficulty of pose and posture estimation of non-cooperative targets and the complexity of FFSM planning and control, the paper proposed an end-to-end target tracking algorithm using deep reinforcement learning (DRL) algorithm. The algorithm uses raw camera images as input and directly outputs the manipulator joint velocities. To deal with the partially observable Markov decision processes (POMDP) problem caused by single image input, we combine the soft actor–critic algorithm with recurrent neural networks. Therefore, the velocity information of the target is introduced to transform the task into a fully observable Markov decision-making process (MDP). Our approach is completely data-driven. It avoids the position and posture estimation of non-cooperative target and the complex modeling motion planning process of FFSM. We trained and tested the algorithm by building a simulation environment in CoppeliaSim. The results showed that our approach achieved good results in active non-cooperative target tracking of FFSM systems and demonstrated the great potential of DRL algorithms in solving space tasks.

show abstract

Reinforcement learning with prior policy guidance for motion planning of dual-arm free-floating space robot

Cited by 11 publications

References 31 publications

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Hybrid-Compliant System for Soft Capture of Uncooperative Space Debris

End-to-end active non-cooperative target tracking of free-floating space manipulators

Contact Info

Product

Resources

About