Online robot introspection via wrench-based action grammars

Abstract

Abstract-Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. Robots under a sense-planact paradigm do not have an additional loop to check their actions. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we seek to enable the robot to recognize and expect its behavior, else detect anomalies. We postulated that noisy wrench data inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is obtained by segmenting and encoding data. And when wrench information represents a sequence of sub-tasks, the vocabulary represents a set of words or sentence and provides a unique identifier. The grammar, which can also include unexpected events, was classified both offline and online for simulated and real robot experiments. Multi-class Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were used to give temporal confidence to the introspection result. Our work's contribution is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or anomalous. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated twoarm experiment. The data set itself is also fully available online and provides a valuable resource by itself for this type of contact task. Our verification mechanism can be used by highlevel planners or reasoning systems to enable intelligent failure recovery or determine the next most optimal manipulation skill to be used. Supplemental information, code, data, and other supporting documentation can be found at [1].

Robot introspection with Bayesian nonparametric vector autoregressive hidden Markov models

“…The sHDP-AR-HMM thus makes stronger generalizations and is a more powerful modeling system than the sHDP-HMM and HMM. Compared to the SVM results in [4], the SVM tends to do better, on the other hand, this system allows for the simultaneous checking of nominal and anomalous skills, not so in [4], where a two stage classifier is used. Having said that, the SVM probabilistic method is able to provide a confidence parameter beyond accuracy classification, something that is not readily available in HMMs, though condition numbers seem possible but were not attempted in this work.…”

“…For contact tasks state estimation, Rojas et al, extract relative-change patterns classified through a small set of categories and aided by contextual information, and where increasingly abstract layers were used to estimate task behaviors [2]. The framework used Bayesian models to provide a belief about its classification [3] and SVMs to identify specific failure modes [4]. In [5], Rodriguez et al, classified wrench data using SVM's to learn a decision rule between successful and failed assemblies offline.…”

A Latent State-Based Multimodal Execution Monitor with Anomaly Detection and Classification for Robot Introspection

“…In Milacski et al [35], the method requires the full data sequence for processing, thus precluding the online detection capability. Besides our previous work [9,36], others have used non-parametric Bayesian approaches to learn HMMs in contact tasks for abnormal detection and recognition. In [37,38], DiLello et al used the sticky hierarchical Dirichlet process prior to learn the model parameters of an HMM based on wrench signatures for an industrial robot alignment task.…”

Recovering from External Disturbances in Online Manipulation through State-Dependent Revertive Recovery Policies

“…Motion's inherent structure is composed of a sequence of primitive or compound skills S m similar to that of language grammar [7], [9], [14], [15]. Just as grammar has rules and order, motion is also organized by rules and order that yield discernible patterns in the sensory-motor action space.…”

“…Their decision's confidence is correlated with outcome success [2]. In robotics, online decision making and robot introspection have begun to receive more attention recently [3]- [9]. The vision is to endow robots with the ability to understand their actions and make timely decisions to achieve their goals and have long-term autonomy.…”