Towards end-to-end formation control for robotic fish via deep reinforcement learning with non-expert imitation

Sun, Yihao; Yan, Chao; Xiang, Xiaojia; Zhou, Han; Tang, Daquan; Zhu, Yi

doi:10.1016/j.oceaneng.2023.113811

Cited by 8 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to bionic action control tasks and motion control tasks, decision making tasks for bionic underwater robots are more diverse, such as searching [ 69 ], obstacle avoidance [ 115 ], formation control [ 116 , 117 ], and other swarm strategies [ 118 , 119 , 120 ]. The majority of current research on RL-based decision making for bionic underwater robots is conducted in simulation environments.…”

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

“…For decentralized circle formation control for fish-like robots, a new MARL method based on value decomposition networks (VDN) was proposed [ 16 ], and the cognitive consistency of multi-agents realized by parameter sharing and the centralized training mechanism with decentralized execution is an important factor in the effective formation of control methods. A dueling double DQN (D3QN)-based approach in the leader–follower topology was proposed for end-to-end formation control [ 117 ], and the blindness of agent exploration at the beginning of training was reduced through imitation learning. Similarly, ref.…”

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

“…To facilitate overcoming difficulties during the training process, more detailed techniques have been summarized as follows. To avoid the blindness of the agent improvement in the early stage of training, an imitation-based action selection strategy [ 117 ] and teaching initialization [ 87 ] are used in RL algorithm training. In order to prevent the RL algorithm from being trapped in the optimum and to accelerate the training, the go-explore strategy in [ 95 ] is used, which records the encountered states into an archive and replays them at the beginning of subsequent episodes.…”

Section: Training and Deployment Methods Of Rl On Bionic Underwater R...mentioning

confidence: 99%

See 2 more Smart Citations

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

View full text Add to dashboard Cite

Bionic robots possess inherent advantages for underwater operations, and research on motion control and intelligent decision making has expanded their application scope. In recent years, the application of reinforcement learning algorithms in the field of bionic underwater robots has gained considerable attention, and continues to grow. In this paper, we present a comprehensive survey of the accomplishments of reinforcement learning algorithms in the field of bionic underwater robots. Firstly, we classify existing reinforcement learning methods and introduce control tasks and decision making tasks based on the composition of bionic underwater robots. We further discuss the advantages and challenges of reinforcement learning for bionic robots in underwater environments. Secondly, we review the establishment of existing reinforcement learning algorithms for bionic underwater robots from different task perspectives. Thirdly, we explore the existing training and deployment solutions of reinforcement learning algorithms for bionic underwater robots, focusing on the challenges posed by complex underwater environments and underactuated bionic robots. Finally, the limitations and future development directions of reinforcement learning in the field of bionic underwater robots are discussed. This survey provides a foundation for exploring reinforcement learning control and decision making methods for bionic underwater robots, and provides insights for future research.

show abstract

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

Section: Training and Deployment Methods Of Rl On Bionic Underwater R...mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

View full text Add to dashboard Cite

show abstract

“…Expanding this concept, Ho J and team introduced the Generative Adversarial Imitation Learning (GAIL) algo-rithm [20], adeptly selecting trajectories closely mirroring human demonstrations, thereby curtailing inefficient maneuvers and enhancing the training process's speed. In a similar vein, Sun Y et al ingeniously melded DQN with behavioral cloning to develop the D3QN algorithm [21], markedly diminishing exploration randomness in initial training phases. Furthermore, Peng X B et al devised the deep mimic approach [22], ingeniously segmenting the reward function into an aggregate of imitation-based exponential components, thereby refining the reinforcement learning process.…”

Section: Introductionmentioning

confidence: 99%

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Zhao,

Ding,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the rapid advancement of industrial automation and artificial intelligence technologies, particularly in the textile industry, robotic technology is increasingly challenged with intelligent path planning and executing high-precision tasks. This study focuses on the automatic path planning and yarn-spool-assembly tasks of textile robotic arms, proposing an end-to-end planning and control model that integrates deep reinforcement learning. The innovation of this paper lies in the introduction of a cascaded fuzzy reward system, which is integrated into the end-to-end model to enhance learning efficiency and reduce ineffective exploration, thereby accelerating the convergence of the model. A series of experiments conducted in a simulated environment demonstrate the model’s exceptional performance in yarn-spool-assembly tasks. Compared to traditional reinforcement learning methods, our model shows potential advantages in improving task success rates and reducing collision rates. The cascaded fuzzy reward system, a core component of our end-to-end deep reinforcement learning model, offers a novel and more robust solution for the automated path planning of robotic arms. In summary, the method proposed in this study provides a new perspective and potential applications for industrial automation, especially in the operation of robotic arms in complex and uncertain environments.

show abstract