Grasp Detection for Robot to Human Handovers Using Capacitive Sensors

Mamaev, Ilshat; Kretsch, David; Alagi, Hosam; Hein, Björn

doi:10.1109/icra48506.2021.9560970

Cited by 6 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When we analyze the impact of each modality (8)(9)(10)(11)(12), we observe that the combination of video and F-T performs best. We next investigate multitask learning with MSTCN by considering only video as input (13)(14)(15)(16)(17). We observe a clear improvement when L H seg is added as a second task.…”

Section: Resultsmentioning

confidence: 93%

“…They find that people expect the robot to provide explanations, and not just non-verbal cues, for why a failure has occurred. In other work [15], [16], specific failures, such as occlusions or no detections of the human's hands, are considered to either disregard an experimental trial, or to abort the handover. Iori et al [17] consider two scenarios which deviate from a nominal handover, in which (i) the person pauses while approaching the robot before continuing, and (ii) the person's arm is pushed by another person during the approach.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Using Visual Anomaly Detection for Task Execution Monitoring

Thoduka

Gall

Plöger

2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.

show abstract

Section: Resultsmentioning

confidence: 93%

Section: Related Workmentioning

confidence: 99%

Using Visual Anomaly Detection for Task Execution Monitoring

Thoduka

Gall

Plöger

2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…These sensors are integrated into the gripper jaws of the robot. To detect a grasp, a threshold value was defined that must be exceeded for the gripper to release the cup ( Mamaev et al, 2021 ). Simultaneously with the opening of the gripper, the robot moves up 10 cm to give the subjects more space to grasp the cup.…”

Section: Methodsmentioning

confidence: 99%

“…Implementing a robot-human handover that is as efficient and fluid as the exchange between humans is considered an open challenge for the robotics research community ( Ortenzi et al, 2021 ). Technological developments are influencing novel ways for robots to function, such as motion profiling, velocity profiling ( Shibata et al, 1995 ; Huber et al, 2008 ), grip force control ( Chan et al, 2013 ), camera-based hand tracking ( Zhang et al, 2020 ) and sensing ( Mamaev et al, 2021 ) combined with machine learning. The goal of all these developments is to achieve the best possible object handover between robots and humans.…”

Section: Introductionmentioning

confidence: 99%

Optimizing human-robot handovers: the impact of adaptive transport methods

et al. 2023

Self Cite

View full text Add to dashboard Cite

Humans are increasingly coming into direct physical contact with robots in the context of object handovers. The technical development of robots is progressing so that handovers can be better adapted to humans. An important criterion for successful handovers between robots and humans is the predictability of the robot for the human. The better humans can anticipate the robot’s actions, the better they can adapt to them and thus achieve smoother handovers. In the context of this work, it was investigated whether a highly adaptive transport method of the object, adapted to the human hand, leads to better handovers than a non-adaptive transport method with a predefined target position. To ensure robust handovers at high repetition rates, a Franka Panda robotic arm with a gripper equipped with an Intel RealSense camera and capacitive proximity sensors in the gripper was used. To investigate the handover behavior, a study was conducted with n = 40 subjects, each performing 40 handovers in four consecutive runs. The dependent variables examined are physical handover time, early handover intervention before the robot reaches its target position, and subjects’ subjective ratings. The adaptive transport method does not result in significantly higher mean physical handover times than the non-adaptive transport method. The non-adaptive transport method does not lead to a significantly earlier handover intervention in the course of the runs than the adaptive transport method. Trust in the robot and the perception of safety are rated significantly lower for the adaptive transport method than for the non-adaptive transport method. The physical handover time decreases significantly for both transport methods within the first two runs. For both transport methods, the percentage of handovers with a physical handover time between 0.1 and 0.2 s increases sharply, while the percentage of handovers with a physical handover time of >0.5 s decreases sharply. The results can be explained by theories of motor learning. From the experience of this study, an increased understanding of motor learning and adaptation in the context of human-robot interaction can be of great benefit for further technical development in robotics and for the industrial use of robots.

show abstract

“…When the hand touches the object, the capacitance spikes and reaches the threshold value, and the gripper receives the signal and releases the object, as shown in Figure 13. Using this method, the monitoring of the whole process from approaching to grasping by hand can be completed to ensure the smooth process of object delivery [14].…”

Section: Handover Things From Robots To Humanmentioning

confidence: 99%

Perspective of Proximity Sensors on Robots Using Cross-Thought

Xing¹

2023

HSET

View full text Add to dashboard Cite

Robots play a significantly important role in the area of industry manufacture. In recent years, with the development of intelligence robots, collaborative robots and human-robot interactive robots, proximity sensors applied on robots become increasingly important and popular, since proximity sensors can assure safety during collaboration process between human and robots and help to complete difficult tasks such as objects grasping, objects handover, etc. This article gives the lasted development of proximity sensors and provides an introduction on proximity sensors which are used on robots by their categories. then some concrete applications of the proximity sensors on robots are introduced, such as collision prevention, objects grasping and handover, landing process control of robot legs and so on. In addition, this article also provides some possible future development directions of proximity sensors, such as mixture usage of different proximity sensors, more application scenarios, improvement of data processing method, etc., which can help future researchers to find research directions quickly.

show abstract

Grasp Detection for Robot to Human Handovers Using Capacitive Sensors

Cited by 6 publications

References 18 publications

Using Visual Anomaly Detection for Task Execution Monitoring

Using Visual Anomaly Detection for Task Execution Monitoring

Optimizing human-robot handovers: the impact of adaptive transport methods

Perspective of Proximity Sensors on Robots Using Cross-Thought

Contact Info

Product

Resources

About