From demonstrations to task-space specifications. Using causal analysis to extract rule parameterization from demonstrations

Abstract

Learning models of user behaviour is an important problem that is broadly applicable across many application domains requiring human–robot interaction. In this work, we show that it is possible to learn generative models for distinct user behavioural types, extracted from human demonstrations, by enforcing clustering of preferred task solutions within the latent space. We use these models to differentiate between user types and to find cases with overlapping solutions. Moreover, we can alter an initially guessed solution to satisfy the preferences that constitute a particular user type by backpropagating through the learned differentiable models. An advantage of structuring generative models in this way is that we can extract causal relationships between symbols that might form part of the user’s specification of the task, as manifested in the demonstrations. We further parameterize these specifications through constraint optimization in order to find a safety envelope under which motion planning can be performed. We show that the proposed method is capable of correctly distinguishing between three user types, who differ in degrees of cautiousness in their motion, while performing the task of moving objects with a kinesthetically driven robot in a tabletop environment. Our method successfully identifies the correct type, within the specified time, in 99% [97.8–99.8] of the cases, which outperforms an IRL baseline. We also show that our proposed method correctly changes a default trajectory to one satisfying a particular user specification even with unseen objects. The resulting trajectory is shown to be directly implementable on a PR2 humanoid robot completing the same task.

Causal Discovery of Dynamic Models for Predicting Human Spatial Interactions

“…Other applications include the use of PCMCI to derive the causal model of an underwater robot trying to reach a target position (Cao et al, 2021) or to predict human spatial interactions in a social robotics context (Castri et al, 2022). Further causality-based approaches can be found in the robot imitation learning and manipulation area (Katz et al, 2018;Angelov et al, 2019;Lee et al, 2021). However, all these solutions rely on a pre-determined set of variables for performing the causal analysis and do not extract the most meaningful ones for the reconstruction of the causal model.…”

Avoiding collaborative paradox in multi‐agent reinforcement learning

“…Despite the great success of RL combined with deep learning on single-agent applications [19], such as video games [20] and traditional complex games [21], there remain new challenges for MARL in real-world applications, such as self-control drones [22], self-driving cars [23], and self-operating robots [24,25]. In multi-agent applications, it requires the following: real-time collaborations, over combinatorial or structured action spaces, sparse rewards, disallowed communications, and incompletely observed information.…”