Deep reinforcement learning for tensegrity robot locomotion

Abstract

Tensegrity robots, composed of rigid rods connected by elastic cables, have a number of unique properties that make them appealing for use as planetary exploration rovers. However, control of tensegrity robots remains a difficult problem due to their unusual structures and complex dynamics. In this work, we show how locomotion gaits can be learned automatically using a novel extension of mirror descent guided policy search (MDGPS) applied to periodic locomotion movements, and we demonstrate the effectiveness of our approach on tensegrity robot locomotion. We evaluate our method with realworld and simulated experiments on the SUPERball tensegrity robot, showing that the learned policies generalize to changes in system parameters, unreliable sensor measurements, and variation in environmental conditions, including varied terrains and a range of different gravities. Our experiments demonstrate that our method not only learns fast, power-efficient feedback policies for rolling gaits, but that these policies can succeed with only the limited onboard sensing provided by SUPERball's accelerometers. We compare the learned feedback policies to learned open-loop policies and hand-engineered controllers, and demonstrate that the learned policy enables the first continuous, reliable locomotion gait for the real SUPERball robot. Our code and other supplementary materials are available from http://rll.berkeley.edu/drl_tensegrity * These authors contributed equally to this work.

Inclined surface locomotion strategies for spherical tensegrity robots

“…Due to the inherent coupled, nonlinear dynamics of the robot, multi-cable actuation policies render robotic control a challenging intellectual task, providing a launch point for future work. We look forward to exploring the integration of artificial intelligence (particularly evolutionary algorithms and deep reinforcement learning architectures) in this robotic platform to optimize locomotive gaits on varied inclines, and even generate optimal tensegrity topologies, areas which have proven promising in prior work [17], [18]. We hope to leverage learning algorithms to achieve more fluid and efficient locomotion using a robust and fully autonomous control policy.…”

Model-Predictive Control With Inverse Statics Optimization for Tensegrity Spine Robots

“…Model-based closed-loop control has been mostly limited to low-dimensional structures [23], [24], [25], [13], [26]. More complex and high dimensional systems have been addressed with model-free methods [16], [22], [18], [27], [28], [29] or open-loop control [30], [31], [32], [20]. In order to use a tensegrity spine with Laika, a model-based closed-loop tracking controller was developed by the authors in [12] and is improved upon in this work.…”

“…(27)(28)(29), in the context of robotics and control systems, did not address these rank issues when implemented for this 2D spine. Reducing (27)(28)(29) to a cablesonly formulation by optimizing only over q s as suggested in [20] only exacerbates these rank issues by defining A with fewer columns. Additionally, the relaxation of this equalityconstrained problem to an inequality-constrained formulation, as used in [20], did not make the problem feasible.…”

“…This rigid body formulation (eqn [32][33][34][35][36][37][38][39][40][41][42][43]. can be combined with the minimum force density optimization problem (eqn [27][28][29]. by substituting the equality constraints, as in…”

A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies

“…Further details are presented in Alg. 2 2) Adversarial transfer of encoder from sim-to-real: Once we have a policy that is performing well in the simulator, we aim to learn an encoder that generates the same distribution of latent states over real images as the pre-trained encoder. To achieve this we begin by freezing the source encoder's learned weights.…”