A Sensorized Multicurved Robot Finger With Data-Driven Touch Sensing via Overlapping Light Signals

Abstract

Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In this paper, we present a multicurved robotic finger with accurate touch localization and normal force detection over complex, three-dimensional surfaces. The key to our approach is the novel use of overlapping signals from light emitters and receivers embedded in a transparent waveguide layer that covers the functional areas of the finger. By measuring light transport between every emitter and receiver, we show that we can obtain a very rich signal set that changes in response to deformation of the finger due to touch. We then show that purely data-driven deep learning methods are able to extract useful information from such data, such as contact location and applied normal force, without the need for analytical models. The final result is a fully integrated, sensorized robot finger, with a low wire count and using easily accessible manufacturing methods, designed for easy integration into dexterous manipulators. P. Piacenza and M. Ciocarlie are with the

A Survey on AI-Driven Digital Twins in Industry 4.0: Smart Manufacturing and Advanced Robotics

“…At the sensing stage, Jin et al reported a smart soft-robotic gripper system based on triboelectric nanogenerator sensors to capture the continuous motion and tactile information for soft gripper control, where PCA and SVM were used to realize real-time prediction [ 260 ]. Data- or AI-driven approaches can also be found in other touch/haptic/force sensing for obtaining better system understanding and task performance [ 280 , 281 , 282 ]. One level higher, at the controller stage, Verner et al implemented online reinforcement learning via a fabricated digital twin, to enable a humanoid robot to lift a weight of unknown mass through autonomous trial-and-error search [ 261 ].…”

Organic Thin Film Transistors in Mechanical Sensors

“…By utilizing the now‐available processing power, sensors and software can extract far more information than in previous eras. This is another pivotal advance that allows for innovations such as the robotic finger touch sensor enabled by neural networks and measuring overlapping optical signals in Piacenza et al [ 161 ] Such an example shows that, by training, many different signals from a host of sensors, not necessarily optical, can be interpreted as a single action or quantity. This processing power also enables more efficient human‐machine interfaces such that researchers can collect, and make sense of, data sets on the human body at large scales.…”

Interpreting and predicting tactile signals for the SynTouch BioTac

“…Fortunately, a large number of compelling tactile sensors have been developed for robotic applications, including the GelSight (Yuan et al, 2017), TacTip (Ward-Cherrier et al, 2018), Soft-bubble (Alspach et al, 2019), DIGIT Lambeta et al (2020), overlapping optical signal sensors (Piacenza et al, 2020), and commercial offerings from SynTouch, Pressure Profile Systems, ATI Industrial Automation, and OnRobot. For recent comprehensive reviews, see Dahiya et al (2010), Yousef et al (2011), Kappassov et al (2015), Chen et al (2018), and Yamaguchi and Atkeson (2019).…”