Static Shape Control of Soft Continuum Robots Using Deep Visual Inverse Kinematic Models

Abstract

Soft continuum robots are highly flexible and adaptable, making them ideal for unstructured environments such as the human body and agriculture. However, their high compliance and manoeuvrability make them difficult to model, sense, and control. Current control strategies focus on Cartesian space control of the end-effector, but few works have explored full-body control. This study presents a novel image-based deep learning approach for closed-loop kinematic shape control of soft continuum robots. The method combines a local inverse kinematics formulation in the image-space with deep convolutional neural networks for accurate shape control that is robust to feedback noise and mechanical changes in the continuum arm. The shape controller is fast and straightforward to implement; it takes only a few hours to generate training data, train the network, and deploy, requiring only a web camera for feedback. This method offers an intuitive and user-friendly way to control the robot's 3D shape and configuration through teleoperation using only 2D hand-drawn images of the desired target state without the need for further user instruction or consideration of the robot's kinematics.

Utilising redundancy in musculoskeletal systems for adaptive stiffness and muscle failure compensation: a model-free inverse statics approach

“…The controller's formulation is the learning-based equivalent of a resolved motion rate controller [79]. Leveraging the generalisability of neural networks and the iterative nature of the approximated pseudoinverse of the local Jacobian, the controller can converge close to desired targets, even in the presence of mechanical changes in the system [99,100]. Consequently, the control system has proven its capacity to compensate for inertial changes in the musculoskeletal system, exhibiting an insignificant reduction in performance compared to the baseline (no weight change), as demonstrated in our weight deviation experiment in section 4.3.2.…”

“…Due to iterative nature of the formulation, which finds the IK solution to the target goal given the systems current posture, the approach is scalable to 3D joint kinematics through the addition of a Z axis component in the state vector. Prior works with kinematically hyper-redundant soft continuum robots using the same learning controller have shown scalability to 3D kinematics [99,100]. Hence, future research will involve implementing and experimenting with the controller in 3D simulations and on a physical robot as an extension of the present study.…”

“…Hence, the inverse mapping would be to find the muscle activation solutions a for a given end-effector pose x. We apply a learning-based inverse control approach [98][99][100], which assumes steady-state conditions. Obtaining the inverse of this function is nontrivial, due to the redundancy of solutions.…”

Model-Free Control of a Soft Pneumatic Segment

“…The problem with data-based approaches is that they do not naturally allow the management of redundancies; however, they often appear in soft robots as a consequence of their high dimensionality. One solution, proposed in [ 76 ], is to work not only with the final tip position but with the whole manipulator. A Convolutional Neural Network (CNN) is employed to read the robot’s current position and generate the pressures necessaries to reach the desired position.…”

Miniaturised Soft Manipulators with Reinforced Actuation Chambers on the Sub-Centimetre Scale

“…Inspired by the STIFF-FLOP design, a two-segment soft instrument was designed in [27], envisioned for cancer imaging and controlled using the Simulation Open Framework Architecture (SOFA), a software specifically developed for implementing FEMbased simulation and control. This soft instrument has a diameter of 11.5 mm with three reinforced chamber pairs [28].…”