CoSTAR: Instructing collaborative robots with behavior trees and vision

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Ratliff

Eppner

et al. 2019

Self Cite

It is difficult to create robust, reusable, and reactive behaviors for robots that can be easily extended and combined. Frameworks such as Behavior Trees are flexible but difficult to characterize, especially when designing reactions and recovery behaviors to consistently converge to a desired goal condition. We propose a framework which we call Robust Logical-Dynamical Systems (RLDS), which combines the advantages of task representations like behavior trees with theoretical guarantees on performance. RLDS can also be constructed automatically from simple sequential task plans and will still achieve robust, reactive behavior in dynamic real-world environments. In this work, we describe both our proposed framework and a case study on a simple household manipulation task, with examples for how specific pieces can be implemented to achieve robust behavior. Finally, we show how in the context of these manipulation tasks, a combination of an RLDS with planning can achieve better results under adversarial conditions.

Section: Related Workmentioning

confidence: 99%

Representing Robot Task Plans as Robust Logical-Dynamical Systems

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Ratliff

Eppner

et al. 2019

Self Cite

“…Communicating control and goals has traditionally been accomplished by specifying high level operations [5], [1], [6], via formal languages like the Problem Domain Description Language (PDDL) [7] or as a Hierarchical Task Network [8]. Such systems provide a straightforward way to compose black-box operations to solve problems.…”

Section: Related Workmentioning

confidence: 99%

“…We collected natural language commands from human annotators through the Mechanical Turk crowd-sourcing platform. 1 Annotators were shown two scene images: one before and one after a block had been stacked on another block. They were instructed to give two distinct commands that would let someone create the second scene from the first ( Figure 5), and were paid $0.25 per such annotation.…”

Section: Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

Prospection: Interpretable plans from language by predicting the future

2019 International Conference on Robotics and Automation (ICRA)

Bisk

Thomason

et al. 2019

Self Cite

High-level human instructions often correspond to behaviors with multiple implicit steps. In order for robots to be useful in the real world, they must be able to to reason over both motions and intermediate goals implied by human instructions. In this work, we propose a framework for learning representations that convert from a natural-language command to a sequence of intermediate goals for execution on a robot. A key feature of this framework is prospection, training an agent not just to correctly execute the prescribed command, but to predict a horizon of consequences of an action before taking it. We demonstrate the fidelity of plans generated by our framework when interpreting real, crowd-sourced natural language commands for a robot in simulated scenes.

“…These developments have led to a growing interest in making it easy for domain experts to transfer knowledge to collaborative robots, either through a user interface [2], [3], [4], [5], natural language [6], or learning from demonstration [7], [8], [9], [10]. To take full advantage of these systems, the human user must have an accurate mental model of a robot's capabilities [11].…”

Section: Introductionmentioning

confidence: 99%

Evaluating Methods for End-User Creation of Robot Task Plans

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Jonathan

Hundt

et al. 2018

Self Cite

How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effective. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as "pick up component A from the right side of the table." Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.