A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Park, Daehyung; Hoshi, Yuuna; Kemp, Charles C.

doi:10.1109/lra.2018.2801475

Cited by 623 publications

(342 citation statements)

References 32 publications

Supporting

Mentioning

283

Contrasting

Unclassified

Order By: Relevance

“…A workshop publication described an early, less-capable version of the meal-assistance system that required fiducial markers placed on the person's head and the bowl [28]. Otherwise, our publications involving meal-assistance have focused on execution monitoring [29,15,30,16]. The newer meal-assistance system that we present now was used in a conference paper [15] to evaluate an execution monitoring system, but the paper provided no details about the meal-assistance system.…”

Section: Our Prior Work On Robot-assisted Feedingmentioning

confidence: 99%

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Park

Hoshi

Mahajan

et al. 2020

Robotics and Autonomous Systems

Self Cite

View full text Add to dashboard Cite

Eating is an essential activity of daily living (ADL) for staying healthy and living at home independently. Although numerous assistive devices have been introduced, many people with disabilities are still restricted from independent eating due to the devices' physical or perceptual limitations. In this work, we present a new meal-assistance system and evaluations of this system with people with motor impairments. We also discuss learned lessons and design insights based on the evaluations. The meal-assistance system uses a general-purpose mobile manipulator, a Willow Garage PR2, which has the potential to serve as a versatile form of assistive technology. Our active feeding framework enables the robot to autonomously deliver food to the user's mouth, reducing the need for head movement by the user. The user interface, visually-guided behaviors, and safety tools allow people with severe motor impairments to successfully use the system. We evaluated our system with a total of 10 able-bodied participants and 9 participants with motor impairments. Both groups of participants successfully ate various foods using the system and reported high rates of success for the system's autonomous behaviors. In general, participants who operated the system reported that it was comfortable, safe, and easy-to-use.

show abstract

Section: Our Prior Work On Robot-assisted Feedingmentioning

confidence: 99%

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Park

Hoshi

Mahajan

et al. 2020

Robotics and Autonomous Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, authors in [28] propose a multi-resolution CNN in the wavelet domain that extracts features independent of phase shifts. Our proposed methodology, instead, leverages a 1D-CNN based Variational AutoEncoder to extract relevant information from the morphology of SCG heartbeats, previously segmented by means of an unsupervised technique; the use of VAE implies a generative model, which may prove useful, e.g., in the context of anomaly detection [29][30][31][32].…”

Section: Related Workmentioning

confidence: 99%

Detection and Analysis of Heartbeats in Seismocardiogram Signals

Mora

Cocconcelli

Matrella

et al. 2020

Sensors

View full text Add to dashboard Cite

This paper presents an unsupervised methodology to analyze SeismoCardioGram (SCG) signals. Starting from raw accelerometric data, heartbeat complexes are extracted and annotated, using a two-step procedure. An unsupervised calibration procedure is added to better adapt to different user patterns. Results show that the performance scores achieved by the proposed methodology improve over related literature: on average, 98.5% sensitivity and 98.6% precision are achieved in beat detection, whereas RMS (Root Mean Square) error in heartbeat interval estimation is as low as 4.6 ms. This allows SCG heartbeat complexes to be reliably extracted. Then, the morphological information of such waveforms is further processed by means of a modular Convolutional Variational AutoEncoder network, aiming at extracting compressed, meaningful representation. After unsupervised training, the VAE network is able to recognize different signal morphologies, associating each user to its specific patterns with high accuracy, as indicated by specific performance metrics (including adjusted random and mutual information score, completeness, and homogeneity). Finally, a Linear Model is used to interpret the results of clustering in the learned latent space, highlighting the impact of different VAE architectural parameters (i.e., number of stacked convolutional units and dimension of latent space).

show abstract

“…To tackle the three challenges, we propose an unsupervised anomaly detection model for event sequence data that builds upon LSTM-based Variational AutoEncoders (VAE) [36]. A recent advance over traditional autoencoder-based anomaly detection techniques, VAE use a probabilistic encoder for modeling the distribution of the latent variables [2].…”

Section: Introductionmentioning

confidence: 99%

Visual Anomaly Detection in Event Sequence Data

Guo

Jin

Chen

et al. 2019

2019 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Fig. 1. The anomaly detection result of the Agavue dataset using ET 3 system. Two anomalous pattern were identified: (1) tool-tips failing to show when users interact with toolbars; (2) users encountering an error after requesting data binding.Abstract-Anomaly detection is a common analytical task that aims to identify rare cases that differ from the typical cases that make up the majority of a dataset. Anomaly detection takes many forms, but is frequently applied to the analysis of event sequence data in addressing real-world problems such as error diagnosis, fraud detection, and vital sign monitoring. With general event sequence data, however, the task of anomaly detection can be complex because the sequential and temporal nature of such data results in diverse definitions and flexible forms of anomalies: a specific event type, a sub-sequence or the entire sequence. This, in turn, increases the difficulty in interpreting detected anomalies, a critical element in raising human confidence with the analysis results. Most prior work in event sequence anomaly detection focuses on detecting only one type of anomaly with a specific application domain. Such approaches do not provide flexible ways of identifying diverse types of anomalies, and provide only limited support for result interpretation. In this paper, we propose an unsupervised anomaly detection algorithm based on Variational AutoEncoders (VAE). The model learns latent representations for all sequences in the dataset and detects anomalies that deviate from the overall distribution. Moreover, the model can estimate an underlying normal progression for each given sequence represented as occurrence probabilities of events along the sequence progression. Events in violation of their occurrence probability (i.e., event occurrences with small occurrence probability, and absent events with large occurrence probability) are identified as abnormal. We also introduce a visualization system, EventThread3, to support interactive exploration of the analysis result. The system facilitates interpretations of anomalies within the context of normal sequence progressions in the dataset through comprehensive one-to-many sequence comparison. Finally, we quantitatively evaluate the performance of our anomaly detection algorithm and demonstrate the effectiveness of our system through case studies in three different application domains and report feedback collected from study participants and expert users.

show abstract

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Cited by 623 publications

References 32 publications

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Detection and Analysis of Heartbeats in Seismocardiogram Signals

Visual Anomaly Detection in Event Sequence Data

Contact Info

Product

Resources

About