Adaptive Actuation of Magnetic Soft Robots Using Deep Reinforcement Learning

Yao, Jianpeng; Cao, Quanliang; Ju, Yuwei; Sun, Yuxuan; Liu, Ruiqi; Han, Xu; Li, Liang

doi:10.1002/aisy.202200339

Cited by 14 publications

(9 citation statements)

References 68 publications

(106 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Yao et al [356] have used RL algorithms for an intelligent approach that tackled the inverse problem of discovering feasible magnetic fields for actuating strip-like soft robots. By employing a helical magnetic hydrogel microrobot, controlled through RL algorithms, the study by Behrens and Ruder [361] demonstrated the capability of the robot to autonomously navigate complex environments.…”

Section: Reinforcement Learning Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Hard magnetics and soft materials—a synergy

Narayanan,

Pramanik,

Arockiarajan

2024

Smart Mater. Struct.

View full text Add to dashboard Cite

Hard-magnetic soft materials (hMSMs) are smart composites that consist of a mechanically soft polymer matrix impregnated with mechanically hard magnetic filler particles. This dual-phase composition renders them with exceptional magneto-mechanical properties that allow them to undergo large reversible deformations under the influence of external magnetic fields. Over the last decade, hMSMs have found extensive applications in soft robotics, adaptive structures, and biomedical devices. However, despite their widespread utility, they pose considerable challenges in fabrication and magneto-mechanical characterization owing to their multi-phase nature, miniature length scales, and nonlinear material behavior. Although noteworthy attempts have been made to understand their coupled nature, the rudimentary concepts of inter-phase interactions that give rise to their mechanical nonlinearity remain insufficiently understood, and this impedes their further advancements. This holistic review addresses these standalone concepts and bridges the gaps by providing a thorough examination of their myriad fabrication techniques, applications, and experimental, and modeling approaches. Specifically, the review presents a wide spectrum of fabrication techniques, ranging from traditional molding to cutting-edge four-dimensional (4D) printing, and their unbounded prospects in diverse fields of research. The review covers various modeling approaches, including continuum mechanical frameworks encompassing phenomenological and homogenization models, as well as microstructural models. Additionally, it addresses emerging techniques like machine learning-based modeling in the context of hMSMs. Finally, the expansive landscape of these promising material systems is provided for a better understanding and prospective research.

show abstract

Section: Reinforcement Learning Modelsmentioning

confidence: 99%

“…Beyond these components, the system involves simulations to demonstrate the design's functionality and performance under various conditions. Reproduced from [356]. CC BY 4.0.…”

Section: Figurementioning

confidence: 99%

Hard magnetics and soft materials—a synergy

Narayanan,

Pramanik,

Arockiarajan

2024

Smart Mater. Struct.

View full text Add to dashboard Cite

show abstract

“…The model is further implemented using the lattice Boltzmann method [50] to simulate fluid-structure interactions, such as a swimming magnetic soft robot. Reduced models of slender hard-magnetic structures, such as shells and plates [51][52][53] as well as rods, [54][55][56][57][58][59] have been developed to further reduce the computational cost. They are very efficient and thus promising for real-time simulations and inverse design of robots.…”

Section: Introductionmentioning

confidence: 99%

“…They are very efficient and thus promising for real-time simulations and inverse design of robots. [57,59] Another challenge is the nonlinear interactions at the interface between the soft robots and their environments, such as friction and adhesion. In many soft robotic systems, [37,[60][61][62][63][64][65][66][67][68] friction and adhesion play a critical role in locomotion performances, especially for those robots that can climb walls, hang from ceilings, and run on granular surfaces.…”

Section: Introductionmentioning

confidence: 99%

Numerical Study of Metachronal Wave‐Modulated Locomotion in Magnetic Cilia Carpets

Jiang

Nelson

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Metachronal motions are ubiquitous in terrestrial and aquatic organisms and have attracted substantial attention in engineering for their potential applications. Hard‐magnetic soft materials are shown to provide new opportunities for metachronal wave‐modulated robotic locomotion by multi‐agent active morphing in response to external magnetic fields. However, the design and optimization of such magnetic soft robots can be complex, and the fabrication and magnetization processes are often delicate and time‐consuming. Herein, a computational model is developed that integrates granular models into a magnetic–lattice model, both of which are implemented in the highly efficient parallel computing platform large‐scale atomic/molecular massively parallel simulator (LAMMPS). The simulations accurately reproduce the deformation of single cilium, the metachronal wave motion of multiple cilia, and the crawling and rolling locomotion of magnetic cilia soft robots. Furthermore, the simulations provide insight into the spatial and temporal variation of friction forces and trajectories of cilia tips. The results contribute to the understanding of metachronal wave‐modulated locomotion and potential applications in the field of soft robotics and biomimetic engineering. The developed model also provides a versatile computational framework for simulating the movement of magnetic soft robots in realistic environments and has the potential to guide the design, optimization, and customization of these systems.

show abstract

“…It requires no prior knowledge and derives optimal policies through trial-and-error interactions with the environment. [13,14] As such, DRL has seen widespread use in novel controller design. [15,16] Bellegarda et al developed a DRL-based approach enabling robust jumping on uneven terrain.…”

Section: Introductionmentioning

confidence: 99%

A Multiobjective Collaborative Deep Reinforcement Learning Algorithm for Jumping Optimization of Bipedal Robot

Tao,

Li,

Cao

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Due to the nonlinearity and underactuation of bipedal robots, developing efficient jumping strategies remains challenging. To address this, a multiobjective collaborative deep reinforcement learning algorithm based on the actor‐critic framework is presented. Initially, two deep deterministic policy gradient (DDPG) networks are established for training the jumping motion, each focusing on different objectives and collaboratively learning the optimal jumping policy. Following this, a recovery experience replay mechanism, predicated on dynamic time warping, is integrated into the DDPG to enhance sample utilization efficiency. Concurrently, a timely adjustment unit is incorporated, which works in tandem with the training frequency to improve the convergence accuracy of the algorithm. Additionally, a Markov decision process is designed to manage the complexity and parameter uncertainty in the dynamic model of the bipedal robot. Finally, the proposed method is validated on a PyBullet platform. The results show that the method outperforms baseline methods by improving learning speed and enabling robust jumps with greater height and distance.

show abstract

Adaptive Actuation of Magnetic Soft Robots Using Deep Reinforcement Learning

Cited by 14 publications

References 68 publications

Hard magnetics and soft materials—a synergy

Hard magnetics and soft materials—a synergy

Numerical Study of Metachronal Wave‐Modulated Locomotion in Magnetic Cilia Carpets

A Multiobjective Collaborative Deep Reinforcement Learning Algorithm for Jumping Optimization of Bipedal Robot

Contact Info

Product

Resources

About