Soft magnetic skin for super-resolution tactile sensing with force self-decoupling

Abstract

Human skin can sense subtle changes of both normal and shear forces (i.e., self-decoupled) and perceive stimuli with finer resolution than the average spacing between mechanoreceptors (i.e., super-resolved). By contrast, existing tactile sensors for robotic applications are inferior, lacking accurate force decoupling and proper spatial resolution at the same time. Here, we present a soft tactile sensor with self-decoupling and super-resolution abilities by designing a sinusoidally magnetized flexible film (with the thickness ~0.5 millimeters), whose deformation can be detected by a Hall sensor according to the change of magnetic flux densities under external forces. The sensor can accurately measure the normal force and the shear force (demonstrated in one dimension) with a single unit and achieve a 60-fold super-resolved accuracy enhanced by deep learning. By mounting our sensor at the fingertip of a robotic gripper, we show that robots can accomplish challenging tasks such as stably grasping fragile objects under external disturbance and threading a needle via teleoperation. This research provides new insight into tactile sensor design and could be beneficial to various applications in robotics field, such as adaptive grasping, dexterous manipulation, and human-robot interaction.

Surface Texture Recognition by Deep Learning‐Enhanced Tactile Sensing

“…As shown in Figure 1A, we design our sensor as an arc‐shaped tip (with radius ≈6 mm) similar to the human fingerpad (with radius ≈8 mm). Compared with our previous design of a planar sensor structure, [ 26 ] such a curved “magnetic skin” could deform more easily to comply with a variety of surface textures (e.g., knitted fabrics and embossed dots patterns of Braille characters) under a small contact force due to the reduced contact area, making the sensor sensitive to the difference among similar textures. At the same time, the curved sensor shape also results in a lower friction force between the sensor and the contact surface during sliding, which could reduce the tactile sensor's wear and test cost and thereby extend its lifespan.…”

“…Here, we propose a novel texture recognition method by designing an arc‐shaped soft tactile sensor (built upon our previous work [ 26 ] ) and an attention‐based long short‐term memory (LSTM) model, which can efficiently recognize both Braille characters and fabrics with high accuracy. Different from traditional tactile sensors that can only characterize the surface texture with a 1D feature, such as resistance, capacitance, or light intensity, our proposed sensor is magnetic‐based and thus can provide 3D feature outputs in terms of the 3‐axis magnetic flux densities, which could convey richer contact information in both normal and shear directions during sliding.…”

A Gradiometric Magnetic Force Sensor Immune to Stray Magnetic Fields for Robotic Hands and Grippers

“…A continuous magnetic skin of 15 mm² (consisting of magnetic microparticles) was demonstrated in [20]. Sub 1-mm "super-resolution" was achieved in [21] by combining an array of magnetic force sensors, sinusoidal magnetization of a flexible film, and deep learning.…”

Review of Robot Skin: A Potential Enabler for Safe Collaboration, Immersive Teleoperation, and Affective Interaction of Future Collaborative Robots

“…As shown in Figure 6, many functional materials are being used in force-or pressure-sensitive skin, such as piezoresistive materials [104]- [108], piezoelectric materials [109]- [112], piezocapacitive materials [113]- [116], triboelectric materials [117]- [120], iontronic materials [121]- [126], magnetic materials [127]- [130], biomimetic materials [131]- [134], and fiber-optic materials [135]- [137]. Table III summarizes the pressure-sensitive robot skin based on the above functional materials.…”

“…than other types of sensors. Existing machine learning methods can be utilized in the sensor calibration for: 1) extending spatial resolution of limited sensing elements [127], [156]; 2) improving the adaptability of mass production long-term usage [157]; compensating hysteresis induced by the viscoelastic property of the polymeric substrate materials [158]; 4) decoupling multimode deformations [135]; and 5) enhancing measurement reliability of large-area sensor array [15], [159] or some specific transductions [160], [161].…”