Sensory-motor augmentation of the robot with shared human perception

Abstract

Robots have replaced people in many manufacturing production lines but the information they gather from sensors might not be sufficient to autonomously accomplish dexterous manipulation operations. Symbiotic human-robot cooperation appears to be a more realistic near future in industrial scenarios. In this paper we present a configuration of human-robot collaboration in which the robot is sensoryaugmented by means of a set of tactile signals coming from the human operator. The incorporation of low-level robot "intelligence" permits the cooperative manipulation of an object while enabling the human operator to stay focused on task itself and carry it out in the most natural way. The effectiveness of this approach is demonstrated in a use case in which a robot helps a human operator to successfully accomplish a writing task. System performance has been evaluated, considering several positions of the tiny vibration sensor in charge of gathering the human perception, by testing it on both the human hand and the co-manipulated object. Results suggest that the sensor provides valuable information for recognizing operator actions when it is placed either on the human hand or on the co-manipulated object. However, the sensor on the finger directly represents the operator's perception, while the output of the sensor attached to the object changes according to the distance between the interaction point and the sensor itself. In addition, in wearing the sensor, neither the object nor the robot need to be instrumented: the operator is free to interact with a large set of objects and collaborate with any existing robot without requiring supplemental equipment. The research leading to these results has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement n • 688857 of the project "SoftPro" and Japan Science and Technology Agency, PRESTO n • JPMJPR14D7.

Shared Haptic Perception for Human-Robot Collaboration

“…The PVDF sensor does not hinder the natural movements of the human hand and allows to directly touch objects, because it is light (about 20 g) and can be worn by wrapping it around one of the fingers, as a ring. In [7], authors showed the advantages of putting the sensor on the human finger, with respect to applying it on the manipulated object. Not only the sensor output "directly represents operator's perception" [7], but instrumenting the human makes it possible to apply the proposed framework in different situations, without having to modify the environment around the user.…”

“…Similarly to [7], to distinguish the different states with the SVM, we used two features: vibration intensity (i RM S ) and frequency ratio (r). They were computed based on the power spectral density (PSD) of the sensor output calculated in the range between f 1 = 100 Hz and f 2 = 1000 Hz 1 :…”

“…Recent advances in interfaces for improved human and robot perception in HRC were surveyed in [5]. On the one hand, human sensorimotor information can be used to monitor human behaviour and plan appropriate robot responses in different phases of a collaborative task [6][7][8]. Peternel et al, for example, used a vision system and EMG electrodes to detect human motion and muscular activity [6], whereas Ishida et al, used a wearable vibration sensor to discriminate human actions [7].…”

“…On the one hand, human sensorimotor information can be used to monitor human behaviour and plan appropriate robot responses in different phases of a collaborative task [6][7][8]. Peternel et al, for example, used a vision system and EMG electrodes to detect human motion and muscular activity [6], whereas Ishida et al, used a wearable vibration sensor to discriminate human actions [7]. On the other hand, visual, auditory, and tactile feedback can be employed to improve human situation awareness in HRC [1,9].…”

“…1. The idea is to integrate the benefits of shared human perception [7] with those of operator awareness [9], creating a bilateral haptic connection between humans and robots, that we call shared haptic perception. The effectiveness of the proposed communication paradigm was demonstrated through an experimental validation involving 8 trained volunteers performing a complex collaborative task with a robot arm.…”