Taking Recoveries to Task: Recovery-Driven Development for Recipe-Based Robot Tasks

Banerjee, Siddhartha; Daruna, Angel; Kent, David; Liu, Weiyu; Balloch, Jonathan; Jain, Abhinav; Krishnan, Akshay; Rana, Muhammad Asif; Ravichandar, Harish; Shah, Binit; Shrivatsav, Nithin; Chernova, Sonia

doi:10.1007/978-3-030-95459-8_36

Cited by 2 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to prior work in robotics [5], we define the robot's action space as the set 𝐴 = {𝑚𝑜𝑣𝑒, 𝑠𝑒𝑔𝑚𝑒𝑛𝑡, 𝑑𝑒𝑡𝑒𝑐𝑡, 𝑓 𝑖𝑛𝑑𝑔𝑟𝑎𝑠𝑝, 𝑔𝑟𝑎𝑠𝑝, 𝑙𝑖 𝑓 𝑡, 𝑝𝑙𝑎𝑐𝑒}, where 𝑚𝑜𝑣𝑒 navigates the robot to a specified location, 𝑠𝑒𝑔𝑚𝑒𝑛𝑡 is used to identify which pixels in its sensory space correspond to objects, 𝑑𝑒𝑡𝑒𝑐𝑡 performs object detection to obtain a label for a given object, 𝑓 𝑖𝑛𝑑𝑔𝑟𝑎𝑠𝑝 executes grasp sampling to identify possible grasp poses for the gripper, 𝑔𝑟𝑎𝑠𝑝 moves the robot arm into a grasp pose and closes the gripper, 𝑙𝑖 𝑓 𝑡 raises the arm, and 𝑝𝑙𝑎𝑐𝑒 places a held object at a specified location.…”

Section: Goal Action Space and State Spacementioning

confidence: 89%

See 1 more Smart Citation

Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery

Das,

Banerjee,

Chernova

2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the growing capabilities of intelligent systems, the integration of robots in our everyday life is increasing. However, when interacting in such complex human environments, the occasional failure of robotic systems is inevitable. The field of explainable AI has sought to make complex-decision making systems more interpretable but most existing techniques target domain experts. On the contrary, in many failure cases, robots will require recovery assistance from non-expert users. In this work, we introduce a new type of explanation, E 𝑒𝑟𝑟 , that explains the cause of an unexpected failure during an agent's plan execution to non-experts. In order for E 𝑒𝑟𝑟 to be meaningful, we investigate what types of information within a set of hand-scripted explanations are most helpful to non-experts for failure and solution identification. Additionally, we investigate how such explanations can be autonomously generated, extending an existing encoder-decoder model, and generalized across environments. We investigate such questions in the context of a robot performing a pick-and-place manipulation task in the home environment. Our results show that explanations capturing the context of a failure and history of past actions, are the most effective for failure and solution identification among non-experts. Furthermore, through a second user evaluation, we verify that our model-generated explanations can generalize to an unseen office environment, and are just as effective as the hand-scripted explanations. CCS CONCEPTS• Human-centered computing → Interaction paradigms.

show abstract

Section: Goal Action Space and State Spacementioning

confidence: 89%

“…1 In our work, 𝑛 ranges from 15 to 20, and 𝑚 = 10. The 𝑚 error timesteps simulate a robot repeatedly attempting to autonomously remedy a failure upon encountering it, reflecting a real world solution to errors where robots try to repeat actions that fail [5].…”

Section: Data Collectionmentioning

confidence: 99%