Pressure distribution classification and segmentation of human hands in contact with the robot body

Abstract

This article deals with the problem of the recognition of human hand touch by a robot equipped with large area tactile sensors covering its body. This problem is relevant in the domain of physical human–robot interaction for discriminating between human and non-human contacts and to trigger and to drive cooperative tasks or robot motions, or to ensure a safe interaction. The underlying assumption used in this article is that voluntary physical interaction tasks involve hand touch over the robot body, and therefore the capability to recognize hand contacts is a key element to discriminate a purposive human touch from other types of interaction. The proposed approach is based on a geometric transformation of the tactile data, formed by pressure measurements associated to a non-uniform cloud of 3D points ( taxels) spread over a non-linear manifold corresponding to the robot body, into tactile images representing the contact pressure distribution in two dimensions. Tactile images can be processed using deep learning algorithms to recognize human hands and to compute the pressure distribution applied by the various hand segments: palm and single fingers. Experimental results, performed on a real robot covered with robot skin, show the effectiveness of the proposed methodology. Moreover, to evaluate its robustness, various types of failures have been simulated. A further analysis concerning the transferability of the system has been performed, considering contacts occurring on a different sensorized robot part.

A Local Filtering Technique for Robot Skin Data

“…Therefore, before discussing about local processing, the concept of neighbourhood or adjacency for taxels must be defined and encoded in a proper data structure. As proposed in [19], a suitable approach to represent taxels positions and their proximity relations is by defining a skin mesh. In this representation, taxels are connected by edges (similarly to a graph structure) which provide both topological and geometric information.…”

“…The proposed technique was compared with a tactile image based approach in the case of the non-planar end-effector. The processing pipeline needed to create a tactile image and summarized at end of Section II was used [19]. The tactile image is generated by resampling the flattened skin geometry with a step of 1 mm, generating an image of 74 × 132 pixels.…”

“…As visible from the picture, the tactile sensing elements are distributed over a curved surface, impeding the ability to directly generate a planar tactile image from this sensors distribution. In our previous work [19], we showed that the application of tactile images is still valid for a robot skin system, after applying a set of geometric transformations creating a flat representation of the skin geometry. Although the method is effective, it requires several processing steps.…”

A Data-Driven Approach for Contact Detection, Classification and Reaction in Physical Human-Robot Collaboration

“…To this aim, ad-hoc hardware devices can be integrated, e.g. [11], or momentum-based observers that only rely on on-board current/torque sensors can be leveraged [7]. For the sake of generality, the latter approach is pursued in this work.…”

“…Other approaches, like the one in [9], leverage the different frequency characteristics of accidental and intentional contacts to achieve classification in the frequency domain. Additional solutions might also leverage proper artificial skins for robots as in [10], [11].…”