Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform

Xu, Zhen; Escalera, Sérgio; Pavao, Adrien; Richard, Magali; Tu, Wei-Wei; Yao, Quanming; Zhao, Huan; Guyon, Isabelle

doi:10.1016/j.patter.2022.100543

Cited by 3 publications

(2 citation statements)

References 15 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It needs to be noted that for a single modality, all I3D variants are the same. When we analyze the impact of each modality (8)(9)(10)(11)(12), we observe that the combination of video and F-T performs best. We next investigate multitask learning with MSTCN by considering only video as input (13)(14)(15)(16)(17).…”

Section: Resultsmentioning

confidence: 99%

“…The types of handover failures in the dataset formed part of the MET-RICS HEART-MET competition at ICRA 2023 as possible behaviours of persons for whom a robot had to fetch an object. The dataset has also been used to create a handover failure detection benchmark hosted on Codabench [9]. We focus on failures that are caused by the human participant in both robot-to-human (R2H) and human-to-robot (H2R)…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Visual Anomaly Detection for Task Execution Monitoring

Thoduka

Gall

Plöger

2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.

show abstract