Learning-Based Visual-Strain Fusion for Eye-in-Hand Continuum Robot Pose Estimation and Control

Abstract

Image processing has significantly extended the practical value of the eye-in-hand camera, enabling and promoting its applications for quantitative measurement. However, fully visionbased pose estimation methods sometimes encounter difficulties in handling cases with deficient features. In this article, we fuse visual information with the sparse strain data collected from a single-core fiber inscribed with fiber Bragg gratings (FBGs) to facilitate continuum robot pose estimation. An improved extreme learning machine algorithm with selective training data updates is implemented to establish and refine the FBG-empowered (Femp) pose estimator online. The integration of F-emp pose estimation can improve sensing robustness by reducing the number of times that visual tracking is lost given moving visual obstacles and varying lighting. In particular, this integration solves pose estimation failures under full occlusion of the tracked features or complete darkness. Utilizing the fused pose feedback, a hybrid controller incorporating kinematics and data-driven algorithms is proposed to accomplish fast convergence with high accuracy. The online-learning error compensator can improve the target tracking performance with a 52.3%-90.1% error reduction compared with Manuscript

A Tensegrity Joint for Low‐Inertia, Compact, and Compliant Soft Manipulators

“…This also provides resistance to torsional force, and helps restore the joint to its original or natural equilibrium. Based on our previous implementations of FBG-based shape sensing, [38,39] an optical fiber with a single core of 18 FBGs (FBGS International) was helically wrapped inside the shallow groove on the soft sheath and sealed using silicone adhesive (Sil-Poxy, Smooth-On Inc.). It functions as a high-level sensor capable of sensing the joint configurations in real time (≥100 Hz) for closed-loop feedback control.…”

“…In our previous work, FBGs helically wrapped on the soft robot surface enabled sensitive shape feedback for reliable closed-loop control. [38,39] In this study, a stretchable silicone sheath embedded with FBGs was integrated as a soft skin protecting our proposed compact tensegrity joint. A learning-based controller with the joint shape feedback and tendon length encoding was constructed.…”