Data-Driven Online Decision Making for Autonomous Manipulation

Abstract

One of the main challenges in autonomous manipulation is to generate appropriate multi-modal reference trajectories that enable feedback controllers to compute control commands that compensate for unmodeled perturbations and therefore to achieve the task at hand. We propose a data-driven approach to incrementally acquire reference signals from experience and decide online when and to which successive behavior to switch, ensuring successful task execution. We reformulate this online decision making problem as a pair of related classification problems. Both process the current sensor readings, composed from multiple sensor modalities, in real-time (at 30 Hz). Our approach exploits that movement generation can dictate sensor feedback. Thus, enforcing stereotypical behavior will yield stereotypical sensory events which can be accumulated and stored along with the movement plan. Such movement primitives, augmented with sensor experience, are called Associative Skill Memories (ASMs). Sensor experience consists of (real) sensors, including haptic, auditory information and visual information, as well as additional (virtual) features. We show that our approach can be used to teach dexterous tasks, e.g. a bimanual manipulation task on a real platform that requires precise manipulation of relatively small objects. Task execution is robust against perturbation and sensor noise, because our method decides online whether or not to switch to alternative ASMs due to unexpected sensory signals.

Fast, robust, and versatile event detection through HMM belief state gradient measures

“…In [6], uses a state-based autoregressive HMM to model the skills and transitions of a task. In [12], two independent naïve Bayes classifiers are run to identify skills and anomalies simultaneously. In [18], multimodal signals were segmented into a grammar via a heuristic.…”

“…Only a couple of these works effect recovery techniques after an anomaly is detected. In [12], the online decision making system is able to recover from external perturbations like human collisions. The recovery however, is performed only once for a single task and no quantitative analysis is provided for the robustness of the identification method postrecovery.…”

Multimodal execution monitoring for anomaly detection during robot manipulation

“…Outside of robotics, researchers have also used the discrete probability distribution over hidden states, which we use to represent execution progress [25], [26]. Kappler et al's recently published method uses multimodal sensing, including sensed forces, audio, and kinematics, to detect failures during robot manipulation [2]. Unlike an HMM, which models state transitions probabilistically, their method assumes that the current state of execution can be determined based on the current multimodal sensor readings alone.…”

“…1) [1], [2]. A robot can monitor this process using a separate system that runs in parallel, which is a form of execution monitoring system (an execution monitor) [3].…”

Learning from Corrective Demonstrations

“…Much prior work has separately investigated the learning and refinement of individual primitives (Akgun et al 2012;Sauser et al 2012;Jain et al 2013;Akgun and Thomaz 2016;Bajcsy et al 2018) and the sequencing of learned primitives (Kappler et al 2015). Recent efforts have focused on learning task models by jointly reasoning over action primitives and their sequencing (Kroemer et al 2015;Niekum et al 2015;Gutierrez et al 2018).…”