Beyond Geometric Path Planning: Learning Context-Driven Trajectory Preferences via Sub-optimal Feedback

The International Journal of Robotics Research

Sharma

Joachims

et al. 2015

Self Cite

116

Abstract-We consider the problem of learning preferences over trajectories for mobile manipulators such as personal robots and assembly line robots. The preferences we learn are more intricate than simple geometric constraints on trajectories; they are rather governed by the surrounding context of various objects and human interactions in the environment. We propose a coactive online learning framework for teaching preferences in contextually rich environments. The key novelty of our approach lies in the type of feedback expected from the user: the human user does not need to demonstrate optimal trajectories as training data, but merely needs to iteratively provide trajectories that slightly improve over the trajectory currently proposed by the system. We argue that this coactive preference feedback can be more easily elicited than demonstrations of optimal trajectories. Nevertheless, theoretical regret bounds of our algorithm match the asymptotic rates of optimal trajectory algorithms.We implement our algorithm on two high degree-of-freedom robots, PR2 and Baxter, and present three intuitive mechanisms for providing such incremental feedback. In our experimental evaluation we consider two context rich settings -household chores and grocery store checkout -and show that users are able to train the robot with just a few feedbacks (taking only a few minutes).

Section: B Feedback Mechanismsmentioning

confidence: 99%

“…Parts of this work has been published at NIPS and ISRR conferences[22,21]. This journal submission presents a consistent full paper, and also includes the proof of regret bounds, more details of the robotic system, and a thorough related work.…”

mentioning

confidence: 99%

Learning preferences for manipulation tasks from online coactive feedback

The International Journal of Robotics Research

Sharma

Joachims

et al. 2015

Self Cite

116

“…We differ from these in that we learn the cost function capturing preferences arising during human-object interactions. Jain et al [32], [7] learned a context-rich cost via iterative feedback from nonexpert users. Similarly, we also learn from the preference data of non-expert users.…”

Section: Related Workmentioning

confidence: 99%

“…The user feedback in PlanIt is in contrast to other learningbased approaches such as learning from the expert's demonstrations (LfD) [14], [15], [16], [39] or the co-active feedback [32], [7]. In both LfD and co-active learning approaches it is time consuming and expensive to collect the preference data on a robotic platform and across many environments.…”

Section: Planit: a Crowdsourcing Enginementioning

confidence: 99%

“…However, most of these works optimize expert-designed cost functions based on different geometric and safety criteria [4], [5], [6]. While satisfying safety criteria is necessary, they alone ignore the contextual interactions in human environments [7]. We take a data driven approach and learn a context-rich cost over the trajectories from the preferences shown by non-expert users.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PlanIt: A crowdsourcing approach for learning to plan paths from large scale preference feedback

2015 IEEE International Conference on Robotics and Automation (ICRA)

Das

Gupta

et al. 2015

Self Cite

We consider the problem of learning user preferences over robot trajectories for environments rich in objects and humans. This is challenging because the criterion defining a good trajectory varies with users, tasks and interactions in the environment. We represent trajectory preferences using a cost function that the robot learns and uses it to generate good trajectories in new environments. We design a crowdsourcing system -PlanIt, where non-expert users label segments of the robot's trajectory. PlanIt allows us to collect a large amount of user feedback, and using the weak and noisy labels from PlanIt we learn the parameters of our model. We test our approach on 122 different environments for robotic navigation and manipulation tasks. Our extensive experiments show that the learned cost function generates preferred trajectories in human environments. Our crowdsourcing system is publicly available for the visualization of the learned costs and for providing preference feedback: http://planit.cs.cornell.edu

Structural-RNN: Deep Learning on Spatio-Temporal Graphs

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Zamir

Savarese

et al. 2016

Self Cite

971

982

Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many problems in computer vision inherently have an underlying high-level structure and can benefit from it. Spatiotemporal graphs are a popular tool for imposing such highlevel intuitions in the formulation of real world problems. In this paper, we propose an approach for combining the power of high-level spatio-temporal graphs and sequence learning success of Recurrent Neural Networks (RNNs). We develop a scalable method for casting an arbitrary spatiotemporal graph as a rich RNN mixture that is feedforward, fully differentiable, and jointly trainable. The proposed method is generic and principled as it can be used for transforming any spatio-temporal graph through employing a certain set of well defined steps. The evaluations of the proposed approach on a diverse set of problems, ranging from modeling human motion to object interactions, shows improvement over the state-of-the-art with a large margin. We expect this method to empower new approaches to problem formulation through high-level spatio-temporal graphs and Recurrent Neural Networks. Links: WebCorresponding Structural-RNN Spatio-temporal graph representation Problem (e.g. Activity)Activity Affordance RNN Activity Affordance Activity Affordance