Multimodal anomaly detection for assistive robots

Park, Daehyung; Kim, Hokeun; Kemp, Charles C.

doi:10.1007/s10514-018-9733-6

Cited by 39 publications

(40 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We control its sensitivity by adjusting the number of support vectors. • HMM-GP: A likelihood-based classifier using an HMM introduced in [32]. We vary the likelihood threshold with respect to the distribution of hidden states.…”

Section: E Baseline Methodsmentioning

confidence: 99%

“…We used data from 1,555 feeding executions collected from 24 able-bodied participants where we newly collected 1,203 non-anomalous feeding executions for this work. 16 participants were male and 8 were female, and the age range was [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35]. We conducted the studies with approval from the Georgia Tech Institutional Review Board (IRB).…”

Section: B Data Collectionmentioning

confidence: 99%

See 1 more Smart Citation

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Park

Hoshi

Kemp

2018

IEEE Robot. Autom. Lett.

Self Cite

622

283

View full text Add to dashboard Cite

The detection of anomalous executions is valuable for reducing potential hazards in assistive manipulation. Multimodal sensory signals can be helpful for detecting a wide range of anomalies. However, the fusion of high-dimensional and heterogeneous modalities is a challenging problem. We introduce a long short-term memory based variational autoencoder (LSTM-VAE) that fuses signals and reconstructs their expected distribution. We also introduce an LSTM-VAE-based detector using a reconstruction-based anomaly score and a state-based threshold. For evaluations with 1,555 robot-assisted feeding executions including 12 representative types of anomalies, our detector had a higher area under the receiver operating characteristic curve (AUC) of 0.8710 than 5 other baseline detectors from the literature. We also show the multimodal fusion through the LSTM-VAE is effective by comparing our detector with 17 raw sensory signals versus 4 hand-engineered features.

show abstract

Section: E Baseline Methodsmentioning

confidence: 99%

Section: B Data Collectionmentioning

confidence: 99%

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Park

Hoshi

Kemp

2018

IEEE Robot. Autom. Lett.

Self Cite

622

283

View full text Add to dashboard Cite

show abstract

“…A workshop publication described an early, less-capable version of the meal-assistance system that required fiducial markers placed on the person's head and the bowl [28]. Otherwise, our publications involving meal-assistance have focused on execution monitoring [29,15,30,16]. The newer meal-assistance system that we present now was used in a conference paper [15] to evaluate an execution monitoring system, but the paper provided no details about the meal-assistance system.…”

Section: Our Prior Work On Robot-assisted Feedingmentioning

confidence: 99%

“…This process can produce large errors due to the noise in the depth data and poor timesynchronization between the RGB and depth images. Thus, our Figure 11: Overview of the multimodal execution monitor, which estimates the task execution state, detect an anomaly [30], and classify the cause of the anomaly [15] for safe feeding assistance.…”

Section: ψ(X) (4)mentioning

confidence: 99%

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Park

Hoshi

Mahajan

et al. 2020

Robotics and Autonomous Systems

Self Cite

View full text Add to dashboard Cite

Eating is an essential activity of daily living (ADL) for staying healthy and living at home independently. Although numerous assistive devices have been introduced, many people with disabilities are still restricted from independent eating due to the devices' physical or perceptual limitations. In this work, we present a new meal-assistance system and evaluations of this system with people with motor impairments. We also discuss learned lessons and design insights based on the evaluations. The meal-assistance system uses a general-purpose mobile manipulator, a Willow Garage PR2, which has the potential to serve as a versatile form of assistive technology. Our active feeding framework enables the robot to autonomously deliver food to the user's mouth, reducing the need for head movement by the user. The user interface, visually-guided behaviors, and safety tools allow people with severe motor impairments to successfully use the system. We evaluated our system with a total of 10 able-bodied participants and 9 participants with motor impairments. Both groups of participants successfully ate various foods using the system and reported high rates of success for the system's autonomous behaviors. In general, participants who operated the system reported that it was comfortable, safe, and easy-to-use.

show abstract

“…Out the system intrusion event. Park et al [11][12][13] carried out an online anomaly monitoring method based on multi-modal sensory signals fusion during robot manipulation, where a parametric HMM (Left-to-right HMM) models a multi-modal time series of a robot performing normal motions, and proposes abnormal thresholds that change by the execution progress to achieve abnormal monitoring. We extend the HMM for anomaly monitoring in the subsequent part of this chapter that inspired by the HMM cases.…”

Section: Related Workmentioning

confidence: 99%

Nonparametric Bayesian Method for Robot Anomaly Monitoring

Zhou

Rojas

et al. 2020

Nonparametric Bayesian Learning for Collaborative Robot Multimodal Introspection

View full text Add to dashboard Cite

In this chapter, we introduce an anomaly monitoring pipeline using the Bayesian nonparametric hidden Markov models after the task representation and skill identification in previous chapter, which divided into three categories according to different thresholds definition, including (i) log-likelihood-based threshold, (ii) threshold based on the gradient of log-likelihood, and (iii) computing the threshold by mapping latent state to log-likelihood. Those method are effectively implement the anomaly monitoring during robot manipulation task. We also evaluate and analyse the performance and results for each method, respectively.

show abstract

Multimodal anomaly detection for assistive robots

Cited by 39 publications

References 69 publications

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

Nonparametric Bayesian Method for Robot Anomaly Monitoring

Contact Info

Product

Resources

About