Hierarchical control of soft manipulators towards unstructured interactions

Abstract

Performing daily interaction tasks such as opening doors and pulling drawers in unstructured environments is a challenging problem for robots. The emergence of soft-bodied robots brings a new perspective to solving this problem. In this paper, inspired by humans performing interaction tasks through simple behaviors, we propose a hierarchical control system for soft arms, in which the low-level controller achieves motion control of the arm tip, the high-level controller controls the behaviors of the arm based on the low-level controller, and the top-level planner chooses what behaviors should be taken according to tasks. To realize the motion control of the soft arm in interacting with environments, we propose two control methods. The first is a feedback control method based on a simplified Jacobian model utilizing the motion laws of the soft arm that are not affected by environments during interaction. The second is a control method based on [Formula: see text]-learning, in which we present a novel method to increase training data by setting virtual goals. We implement the hierarchical control system on a platform with the Honeycomb Pneumatic Networks Arm (HPN Arm) and validate the effectiveness of this system on a series of typical daily interaction tasks, which demonstrates this proposed hierarchical control system could render the soft arms to perform interaction tasks as simply as humans, without force sensors or accurate models of the environments. This work provides a new direction for the application of soft-bodied arms and offers a new perspective for the physical interactions between robots and environments.

Surprisingly Robust In-Hand Manipulation: An Empirical Study

“…In the domain of robotic manipulation compliant hands like the Pisa/IIT SoftHand 2 [11], the RBO Hand 2 [10], the i-Hy Hand [26] as well as the dexterous gripper presented in [27] have shown to be well suited for grasping and inhand manipulation. Besides hands, the dynamic properties of a system composed of a soft arm with a soft gripper can be used to robustly solve various interaction tasks [28].…”

A Survey for Machine Learning-Based Control of Continuum Robots

“…The control policy was learned using Q-learning with the simulation data, demonstrating its effectiveness and robustness in simulation and practice. Jiang et al (2021) adopted the same soft arm and developed a hierarchical control algorithm for complex tasks such as opening a drawer and rotating a handwheel. The control architecture was inspired by human decision-making process.…”

“…In the gradient-based approach, the policy function is maximized with gradient-descent iteratively (Thuruthel et al, 2018;Liu et al, 2020). In contrast to policy search reinforcement learning, valuebased methods generate the optimal control policy by optimizing the value function, including SARSA (Ansari et al, 2017b), Q-learning (You et al, 2017;Jiang et al, 2021), DQN (Satheeshbabu et al, 2019;Wu et al, 2020) and its various extensions (e.g., DDQN (You et al, 2019) and Double DQN). The actor-critic approach is a combination of policy-based and value-based reinforcement learning, where the actor executes referring to the policy; thereby the critic calculates the value function to evaluate the actor (Satheeshbabu et al, 2020).…”

“…Different from rigid-link mechanisms which have definite kinematic mapping, soft robots meet great challenges in accurate analytical modeling, considering the nonlinear deformation induced by the actuation, material elasticity, and susceptibility to contacts with surroundings. Generally speaking, the actuation mechanism of continuum robots (Rolf and Steil, 2013;Yip and Camarillo, 2014;Giorelli et al, 2015a;Ansari et al, 2017a;Lee et al, 2017a;Satheeshbabu et al, 2019;Jiang et al, 2021) can be categorized as intrinsic and extrinsic (Robinson and Davies, 1999), based on the location of the actuator. The intrinsic mechanism means that the actuators are located inside and form as part of the mechanism (Fu et al, 2020).…”

“…Their effectiveness in the feed-forward loop could not be guaranteed, since the analytic shape reconstructed from the set of possible length combinations or other configuration parameters is unknown and non-stationary FIGURE 1 | Continuum robots driven by (A-E) pneumatics or (F-G) tendons. Image sources for (A)-(G) in sequence: (Jiang et al, 2016;Satheeshbabu et al, 2019;Ansari et al, 2017a;Rolf and Steil, 2013;Jiang et al, 2021;Giorelli et al, 2015a;Yip and Camarillo, 2014). (Nordmann et al, 2012).…”