TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

Abstract

Vision-based tactile sensors (VBTSs) have become the de facto method for giving robots the ability to obtain tactile feedback from their environment. Unlike other solutions to tactile sensing, VBTSs offer high spatial resolution feedback without compromising on instrumentation costs or incurring additional maintenance expenses. However, conventional cameras used in VBTS have a fixed update rate and output redundant data, leading to computational overhead.In this work, we present a neuromorphic vision-based tactile sensor (N-VBTS) that employs observations from an event-based camera for contact angle prediction. In particular, we design and develop a novel graph neural network, dubbed TactiGraph, that asynchronously operates on graphs constructed from raw N-VBTS streams exploiting their spatiotemporal correlations to perform predictions. Although conventional VBTSs use an internal illumination source, TactiGraph is reported to perform efficiently in both scenarios (with and without an internal illumination source) thus further reducing instrumentation costs. Rigorous experimental results revealed that TactiGraph achieved a mean absolute error of 0.62∘ in predicting the contact angle and was faster and more efficient than both conventional VBTS and other N-VBTS, with lower instrumentation costs. Specifically, N-VBTS requires only 5.5% of the computing time needed by VBTS when both are tested on the same scenario.

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

“…The filtering analysis of pulse signal is a preprocessing technique from the perspective of signal processing, and it is also the application basis of the visual analysis task of neuromorphic vision sensor. Sajwani et al (Sajwani et al, 2023) proposed a general filtering method for asynchronous spatiotemporal pulse signals, which consists of hierarchical filtering in the time domain or spatial domain, which can be extended to complex non-linear filters such as edge detection. Linares et al (Linares et al, 2019) filter noise reduction and horizontal feature extraction of asynchronous spatiotemporal pulse signals on FPGA, which can significantly improve target recognition and tracking performance.…”