Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects

Abstract

Humans display the remarkable ability to sense the world through tools and other held objects. For example, we are able to pinpoint impact locations on a held rod and tell apart different textures using a rigid probe. In this work, we consider how we can enable robots to have a similar capacity, i.e., to embody tools and extend perception using standard grasped objects. We propose that vibro-tactile sensing using dynamic tactile sensors on the robot fingers, along with machine learning models, enables robots to decipher contact information that is transmitted as vibrations along rigid objects. This paper reports on extensive experiments using the BioTac micro-vibration sensor and a new event dynamic sensor, the NUSkin, capable of multitaxel sensing at 4 kHz. We demonstrate that fine localization on a held rod is possible using our approach (with errors less than 1 cm on a 20 cm rod). Next, we show that vibro-tactile perception can lead to reasonable grasp stability prediction during object handover, and accurate food identification using a standard fork. We find that multi-taxel vibro-tactile sensing at sufficiently high sampling rate (above 2 kHz) led to the best performance across the various tasks and objects. Taken together, our results provides both evidence and guidelines for using vibrotactile perception to extend tactile perception, which we believe will lead to enhanced competency with tools and better physical human-robot-interaction.

Boost event-driven tactile learning with location spiking neurons

“…With the prevalence of material science and robotics, several tactile sensors have been developed, including non-event-based tactile sensors like the iCub RoboSkin (Schmitz et al, 2010) and the SynTouch BioTac (Fishel and Loeb, 2012) and event-driven tactile sensors like the NeuTouch (Taunyazoz et al, 2020) and the NUSkin (Taunyazov et al, 2021). In this paper, we focus on event-driven tactile learning with SNNs.…”

Understanding Dynamic Tactile Sensing for Liquid Property Estimation

“…They can recognize the surfaces with 80% accuracy based on spectro-temporal features. Taunyazov et al [18] used dynamic tactile sensing to achieve grasp stability prediction during object handover and accurate food identification through tools. They also showed the ability to accurately localize taps on an acrylic rod based on vibrations caused by the tapping.…”

Event - Driven Tactile Learning with Location Spiking Neurons

“…• An example for the arch-like location order: [11,25,35 • An example for the whorl-like location order: [21,15,16,23,27,24,17,6,9,12,20,29,33,34,31,28,22,14,10,1,2,7,18,30,37,39,38,32,19,8,3,4,11,25,35,36,26,13,5] • An example for the loop-like location order: [1,2,3,4,5,6,7,8,9,10,11,…”

“…With the prevalence of material science and robotics, several tactile sensors have been developed, including non-eventbased tactile sensors like the iCub RoboSkin [33] and the SynTouch BioTac [34] and event-driven tactile sensors like the NeuTouch [11] and the NUSkin [35]. In this paper, we focus on event-driven tactile learning with SNNs.…”

“…• An example for the arch-like location order: [11,25,35 2) Hybrid SRM FC: Similar to the spike-count loss of prior works [11], [14], we propose a location spike-count loss to optimize the SNN with LSRM neurons:…”