A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications

Abstract

In this paper, a novel Vision-Based Measurement (VBM) approach is proposed to estimate the contact force and classify materials in a single grasp. This approach is the first event-based tactile sensor which utilizes the recent technology of neuromorphic cameras. This novel approach provides a higher sensitivity, a lower latency, and less computational and power consumption compared to other conventional visionbased techniques. Moreover, Dynamic Vision Sensor (DVS) has a higher dynamic range which increases the sensor sensitivity and performance in poor lighting conditions. Two time-series machine learning methods, namely, Time Delay Neural Network (TDNN) and Gaussian Process (GP) are developed to estimate the contact force in a grasp. A Deep Neural Network (DNN) is proposed to classify the object materials. Forty-eight experiments are conducted for four different materials to validate the proposed methods and compare them against a piezoresistive force sensor measurements. A leave-one-out cross-validation technique is implemented to evaluate and analyze the performance of the proposed machine learning methods. The contact force is successfully estimated with a mean squared error of 0.16 N and 0.17 N for TDNN and GP respectively. Four materials are classified with an average accuracy of 79.17% using unseen experimental data. The results show the applicability of eventbased sensors for grasping applications.

Dynamic-Vision-Based Force Measurements Using Convolutional Recurrent Neural Networks

“…Similar to [22], only non-zero measurements are considered to provide a realistic assessment of the sensor. Finally, we train and perform the same experiments for TDNN network to compare the proposed methods with our previous work [13]. Table 2 presents the average of Mean Absolute Error (MAE) and MSE over all the folds where the standard deviation is denoted as σ.…”

“…Similarly, new approaches for slip detection with the DVS are investigated in [26][27][28]. In another work, we proposed a novel framework based on the Dynamic Vision Sensor (DVS) to acquire force magnitude and classify materials in a grasp [13]. Two machine learning methods, TDNN and Gaussian Process (GP), were implemented to estimate the contact force for the objects with same shape and size.…”

“…Polarity is defined in a binary format which is either 0 or 1 for negative and positive polarities respectively, as shown in Figure 1. In [13], the events are accumulated over time without consideration of their position information to estimate the contact force. In this work, an event-framing technique is considered to convert raw events into sequential frames.…”

“…This approach enables the sensor to estimate the contact force accurately even for the holding phase whereas a slight vibration occurs. Moreover, the sensor latency is reduced to 10 ms (one frame) due to the networks architectures which is 52% less compared to our previous work using Time Delayed Neural Networks (TDNN) with 21 ms delay [13]. The main application of the proposed sensor is to estimate the applied force while grasping hard objects with a flat surface repeatedly.…”

“…In [13], we proposed a dynamic-vision-based tactile sensor to estimate the contact force for objects with the same size during the grasping and releasing phases using Time Delay Neural Network (TDNN). However, objects with different size have a variant contact area with the silicone membrane which requires a complex dynamic method to relate events and force measurements at each time stamp.…”