Probabilistic Multimodal Modeling for Human-Robot Interaction Tasks

Abstract

Human-robot interaction benefits greatly from multimodal sensor inputs as they enable increased robustness and generalization accuracy. Despite this observation, few HRI methods are capable of efficiently performing inference for multimodal systems. In this work, we introduce a reformulation of Interaction Primitives which allows for learning from demonstration of interaction tasks, while also gracefully handling nonlinearities inherent to multimodal inference in such scenarios. We also empirically show that our method results in more accurate, more robust, and faster inference than standard Interaction Primitives and other common methods in challenging HRI scenarios.

Learning Whole-Body Human-Robot Haptic Interaction in Social Contexts

“…A. Preliminaries: Bayesian Interaction Primitives [2], [3] We define an interaction Y as a time series of Ddimensional sensor observations over time, Y 1:T = [y 1 , . .…”

“…Following the ensemble variant of BIP, ensemble Bayesian Interaction Primitives [3], the posterior density in Eq. ( 1) is approximated with a Monte Carlo ensemble which is updated as a two-step recursive filter [19]: the prediction step to propagate each sample forward in time according to a constant velocity state transition function g(•) with process noise Q:…”

“…We model the interaction as a Bayesian Interaction Primitive (BIP) [2], [3], [4], a spatiotemporal LfD framework. This model is capable of predicting both an appropriate robotic response (consisting of joint trajectories) as well as contact forces that should be exerted by the robot, given observations of the partner's pose and the force currently exerted by the partner onto the robot.…”

“…The predicted joint trajectories and contact forces are then sent to a motion retargeting controller [5], which is capable of accounting for variances in the partner's body shape and size. While previous applications of this framework include haptic interaction [3], the tactile information has thus far been dense, low-dimensional, and only used as an input observation.…”

User intent estimation during robot learning using physical human robot interaction primitives

“…A probabilistic approach to trajectory generation and user intent estimation are popular due to probabilistic properties of the model, with notable applications for HRI utilizing Interaction Primitives (IP) and its extensions: e.g. human robot gestures (Ben Amor et al 2014), collaborative object covering (Cui et al 2019), and hand shaking (Campbell et al 2019). The probabilistic model embeds the user intent in their outputs, making it suitable to utilize probabilistic operators to adapt the model for goal and trajectory adaptation (Bajcsy et al 2017;Koert et al 2019), sequential intent estimation (Matsubara et al 2015), and stiffness adaptation (Rozo et al 2016).…”