Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction

Abstract

This paper presents a method for constructing human-robot interaction policies in settings where multimodality, i.e., the possibility of multiple highly distinct futures, plays a critical role in decision making. We are motivated in this work by the example of traffic weaving, e.g., at highway onramps/off-ramps, where entering and exiting cars must swap lanes in a short distance-a challenging negotiation even for experienced drivers due to the inherent multimodal uncertainty of who will pass whom. Our approach is to learn multimodal probability distributions over future human actions from a dataset of human-human exemplars and perform real-time robot policy construction in the resulting environment model through massively parallel sampling of human responses to candidate robot action sequences. Direct learning of these distributions is made possible by recent advances in the theory of conditional variational autoencoders (CVAEs), whereby we learn action distributions simultaneously conditioned on the present interaction history, as well as candidate future robot actions in order to take into account response dynamics. We demonstrate the efficacy of this approach with a human-in-theloop simulation of a traffic weaving scenario. , x (t+i+1) ) = J c +J a +J l +J d corresponding to collision avoidance, control effort, lane change incentive, and longitudinal disambiguation incentive defined as: J c = 1000 · 1 {|∆s|<8∧|∆τ |<2} · (9.25 − ∆s 2 + ∆τ 2 ) J a =s 2 r J l = −500 · min(1.5 + s r /150, 1) · |τ r − τ goal | J d = −100 · min(max(∆s∆ṡ, 0), 1)

On the Utility of Model Learning in HRI

“…The Theory-of-Mind method models the human's actions as explicitly optimizing some cost function. An alternative is to learn this function directly from data via, e.g., a conditional neural network, as in [17]. Vanilla.…”

“…But even if humans do use such tools in interaction, it is not at all clear that robots ought to as well. For robots and interaction, methods from all three paradigms exist: model-free [14]- [16], regular modelbased [17], and ToM-based [18]- [22]. Ideally, we'd want to settle the debate for HRI by seeing what works best in practice.…”

Cooperation-Aware Reinforcement Learning for Merging in Dense Traffic

“…improvement in performance can be gained when the planning algorithm has access to information about the driver internal state in lane changing scenarios. Previous work address the problem of modeling interactions between traffic participants using data-driven approaches, probabilistic models, inverse reinforcement learning, rule-based methods, or game theoretic frameworks [2], [4], [5], [8], [9]. Inverse reinforcement learning techniques and game theoretic frameworks are generally too computationally expensive to be used in an online planning algorithm considering more than two traffic participants [5], [9].…”

“…Inverse reinforcement learning techniques and game theoretic frameworks are generally too computationally expensive to be used in an online planning algorithm considering more than two traffic participants [5], [9]. Schmerling et al demonstrated a datadriven approach to learn the interaction model on a traffic weaving scenario involving two agents [4]. They leveraged parallelization to use this model efficiently for online planning.…”

On Infusing Reachability-Based Safety Assurance Within Probabilistic Planning Frameworks for Human-Robot Vehicle Interactions

“…Our intent is to enable truly minimal intervention against the direction of a higher-level planner when evasive action is required. Second, we evaluate the benefits and performance of this safe control methodology in the context of a probabilistic planning framework for the traffic weaving scenario studied at a high level in [11], wherein two cars, initially side-by-side, must swap lanes in a limited amount of time and distance. Experiments with a full-scale steer-by-wire vehicle reveal that our com- Fig.…”