Bio-Inspired Proprioceptive Touch of a Soft Finger with Inner-Finger Kinesthetic Perception

Abstract

In-hand object pose estimation is challenging for humans and robots due to occlusion caused by the hand and object. This paper proposes a soft finger that integrates inner vision with kinesthetic sensing to estimate object pose inspired by human fingers. The soft finger has a flexible skeleton and skin that adapts to different objects, and the skeleton deformations during interaction provide contact information obtained by the image from the inner camera. The proposed framework is an end-to-end method that uses raw images from soft fingers to estimate in-hand object pose. It consists of an encoder for kinesthetic information processing and an object pose and category estimator. The framework was tested on seven objects, achieving an impressive error of 2.02 mm and 11.34 degrees for pose error and 99.05% for classification.

Anthropomorphic Soft Hand: Dexterity, Sensing, and Machine Learning

“…This deficiency hinders the ability to regulate contact forces accurately and execute precise manipulations [113]. Substantial efforts have been directed towards developing external perception capabilities for soft hands, with tactile sensing identified as crucial for safe, refined object grasping and manipulation [114,115].…”

“…Inspired by proprioception, embedded cameras and deep learning architectures have been proposed for object recognition in soft fingers. Liu et al presented a soft finger design that integrated inner vision and kinesthetic sensing to replicate the object pose estimation capabilities observed in human fingers [115]. The proposed framework offered a comprehensive solution that leveraged raw images captured by the soft fingers to estimate the pose of objects held in the hand, providing an end-to-end approach for accurate object pose estimation.…”