On the Utility of Model Learning in HRI

Choudhury, Rohan; Swamy, Gokul; Hadfield-Menell, Dylan; Dragan, Anca D.

doi:10.1109/hri.2019.8673256

Cited by 41 publications

(32 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…tion Learning or Behavioral Cloning 47 is an attempt to mimic the policy of another human or agent. On the other hand, Inverse Reinforcement Learning 48,49 aims to recover reward functions. Modeling the impact of other agents have also been shown to be useful to stabilize the training process for multi-agent reinforcement learning 50 .…”

Section: Modelling Other Agents Several Studies Have Focused On Modementioning

confidence: 99%

Visual behavior modelling for robotic theory of mind

Chen

Vondrick

Lipson

2021

Sci Rep

View full text Add to dashboard Cite

Behavior modeling is an essential cognitive ability that underlies many aspects of human and animal social behavior (Watson in Psychol Rev 20:158, 1913), and an ability we would like to endow robots. Most studies of machine behavior modelling, however, rely on symbolic or selected parametric sensory inputs and built-in knowledge relevant to a given task. Here, we propose that an observer can model the behavior of an actor through visual processing alone, without any prior symbolic information and assumptions about relevant inputs. To test this hypothesis, we designed a non-verbal non-symbolic robotic experiment in which an observer must visualize future plans of an actor robot, based only on an image depicting the initial scene of the actor robot. We found that an AI-observer is able to visualize the future plans of the actor with 98.5% success across four different activities, even when the activity is not known a-priori. We hypothesize that such visual behavior modeling is an essential cognitive ability that will allow machines to understand and coordinate with surrounding agents, while sidestepping the notorious symbol grounding problem. Through a false-belief test, we suggest that this approach may be a precursor to Theory of Mind, one of the distinguishing hallmarks of primate social cognition.

show abstract

Section: Modelling Other Agents Several Studies Have Focused On Modementioning

confidence: 99%

Visual behavior modelling for robotic theory of mind

Chen

Vondrick

Lipson

2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…If slowing down is necessary for collision avoidance or because a stop sign is coming up, that is what the planner will do. And indeed, prior work has shown such predictors to be preferable in highly interactive domains, depending on how one collects their training data [4].…”

Section: Test Datamentioning

confidence: 99%

On complementing end-to-end human behavior predictors with planning

Sun¹,

Jia²,

Dragan³

2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

High capacity end-to-end approaches for human motion (behavior) prediction have the ability to represent subtle nuances in human behavior, but struggle with robustness to out of distribution inputs and tail events. Planningbased prediction, on the other hand, can reliably output decent-but-not-great predictions: it is much more stable in the face of distribution shift (as we verify in this work), but it has high inductive bias, missing important aspects that drive human decisions, and ignoring cognitive biases that make human behavior suboptimal. In this work, we analyze one family of approaches that strive to get the best of both worlds: use the end-to-end predictor on common cases, but do not rely on it for tail events / out-of-distribution inputsswitch to the planning-based predictor there. We contribute an analysis of different approaches for detecting when to make this switch, using an autonomous driving domain. We find that promising approaches based on ensembling or generative modeling of the training distribution might not be reliable, but that there very simple methods which can perform surprisingly well -including training a classifier to pick up on tell-tale issues in predicted trajectories.

show abstract

“…following a Gaussian distribution with covariance matrix Σ = diag(0.75, 0.75). We are interested in the task of steering the robot into a goal set [5,7] × [5,7] To construct the abstraction-based controller, we partition the state space with discretization parameters (0.5, 0.5), and the input space with (0.1, 0.1). This leads to a total number of states equal to |𝑋 | = 1600 and a number of inputs equal to |𝑈 | = 441 (by including the upper and lower limits of the input space as additional input) leading to a complexity of |𝑋 × 𝑈 | = 705600.…”

Section: Benchmarks and Performancementioning

confidence: 99%

“…Third, the use of neural networks to guide the design of the abstraction-based controller opens the door to encode the human's preferences for how a dynamical system should act. Such human's preference is crucial for several real-world settings in which a human user or operator interacts with an autonomous dynamical system [5]. Current research found that human preferences can be efficiently captured using expert demonstrations and preference-based learning which can be hard to be accurately capture in the form of a logical formulae or a reward function [10].…”

Section: Introductionmentioning

confidence: 99%

NNSynth: Neural Network Guided Abstraction-Based Controller Synthesis for Stochastic Systems

Sun¹,

Shoukry²

2021

Preprint

View full text Add to dashboard Cite

In the last decade, abstraction-based controller synthesis techniques have gained considerable attention due to their ability to synthesize correct-by-design feedback controllers from high-level specifications. Nevertheless, a significant drawback of these techniques is the need to explore large spaces of quantized state and input spaces to find a controller that satisfies the specification. On the contrary, recent advances in machine learning, in particular imitation learning and reinforcement learning, paved the way to several techniques to design controllers (or policies) for highly nonlinear systems with large state and input spaces, albeit their lack of rigorous correctness guarantees. This motivates the question of how to use machine learning techniques to guide the synthesis of abstraction-based controllers. In this paper, we introduce NNSynth, a novel tool for abstraction-based controller synthesis. Unique to NNSynth is the use of machine learning techniques to guide the search over the space of controllers to find candidate Neural Network (NN)-based controllers. Next, these NNs are "projected" and the control actions that are close to the NN outputs are used to construct a "local" abstraction for the system. An abstraction-based controller is then synthesized from such a "local" abstract model. If a controller that satisfies the specifications is not found, then the best found controller is "lifted" to a neural network controller for additional training. Our experiments show that this neural networkguided synthesis leads to more than 50× or even 100× speedup in high dimensional systems compared to the state-of-the-art.

show abstract

On the Utility of Model Learning in HRI

Cited by 41 publications

References 26 publications

Visual behavior modelling for robotic theory of mind

Visual behavior modelling for robotic theory of mind

On complementing end-to-end human behavior predictors with planning

NNSynth: Neural Network Guided Abstraction-Based Controller Synthesis for Stochastic Systems

Contact Info

Product

Resources

About