Human intention inference through interacting multiple model filtering

Ravichandar, Harish; Dani, Ashwin P.

doi:10.1109/mfi.2015.7295812

Cited by 20 publications

(11 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This inferred both the target object a user was reaching for, as well as detecting the completion of the user reaching motion. The end of the reaching motion was used to switch to the next assembly step [17]. A limitation of G-MMIE is that it is based on a definition of 'action' that is limited to just the reaching event, and so it does not account for its environmental consequence (such as grasping versus not grasping an object and therefore switching to the next procedural steps vs remaining in the same step).…”

Section: Intent Prediction Over Multiple Stepsmentioning

confidence: 99%

Detecting errors in pick and place procedures

Bovo

Binetti

Brumby

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Many human activities, such as manufacturing and assembly, are sequence-constrained procedural tasks (SPTs): they consist of a series of steps that must be executed in a specific spatial/temporal order. However, these tasks can be error prone -steps can be missed out, executed out-of-order, and repeated. The ability to automatically predict if a person is about to commit an error could greatly help in these cases. The prediction could be used, for example, to provide feedback to prevent mistakes or mitigate their effects. In this paper, we present a novel approach for real-time error prediction for multi-step sequence tasks which uses a minimum viable set of behavioural signals. We have three main contributions. The first we present an architecture for real-time error prediction based on task tracking and intent prediction. The second is to explore the effectiveness of using hand position and eye-gaze tracking for task tracking. We confirm that eye-gaze is more effective for intent prediction, hand tracking is more accurate for task tracking and that combining the two provides the best overall response. We show that using Hands and Gaze tracking data we can predict selection/placement errors with an F1 score of 97%, approximately 300ms before the error would occur. Finally, we discuss the application of this hand-gaze error detection architecture used in conjunction with head-mounted AR displays, to support industrial manual assembly.

show abstract

Section: Intent Prediction Over Multiple Stepsmentioning

confidence: 99%

Detecting errors in pick and place procedures

Bovo

Binetti

Brumby

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

show abstract

“…(3) the Bayesian model, in which a human is treated as a stochastic agent [138], and a Bayesian network [110,117] is associated to his decisions, i.e. the human's motion follows a probabilistic distribution conditioned on the robot's behavior.…”

Section: Design Of the Knowledgementioning

confidence: 99%

Designing Robot Behavior in Human-Robot Interactions

Liu¹,

Tian²,

Lin³

et al. 2019

View full text Add to dashboard Cite

Designing Robot Behavior in Human-Robot InteractionsHuman-robot interactions (HRI) have been recognized to be a key element of future robots in many application domains such as manufacturing, transportation, service and entertainment. These applications entail huge social and economical impacts. Future robots are envisioned to function as human's counterparts, which are independent entities that make decisions for themselves; intelligent actuators that interact with the physical world; and involved observers that have rich senses and critical judgements. Most importantly, they are entitled social attributions to build relationships with humans. We call these robots corobots.Technically, it is challenging to design the behavior of co-robots. Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. The fundamental research question to address in this dissertation is how to ensure that co-robots operate efficiently and safely in dynamic uncertain environments.The focus of this dissertation is 1) to set up a unified analytical framework for various human-robot systems; 2) to establish a methodology to design the robot behavior to address the fundamental problem.A multi-agent framework to model human-robot systems is introduced in Chapter 2. In order to address the uncertainties during human-robot interactions, a unique parallel planning and control architecture is introduced in Chapter 2, which has a cognition module for human behavior estimation and human motion prediction, a long term global planner to ensure efficiency of robot behavior, and a short term local planner to ensure real time safety under uncertainties. The functionalities of these components are discussed in Chapter 3 to Chapter 5. Chapter 3 discusses the cognition module, which includes offline classification and online adaptation of various human behaviors. Chapter 4 and Chapter 5 discuss the optimal control or optimization problems for short term and long term robot motion planning. In a cluttered environment, the optimization problems are highly nonlinear and non-convex, hence hard to solve in real time, which may delay the robot's response in emergency situations. Fast online algorithms are developed to handle the issue: the convex feasible setThe past five years has been an incredible journey in my life. I would not have come this far without the tremendous help from many people.My deep and sincere gratitude goes to my Ph.D. advisor Professor Masayoshi Tomizuka, who has been a great mentor to me during the past five years for his profound knowledge, insightful visions, enthusiasm on work, and humor in life. Moreover, Professor Tomizuka respects all my research ideas, inspires me to explore the unknowns, and offers the strongest support in my career development. I sincerely wish I could be a great person, an extraordinary mentor as he is in the future.

show abstract

“…Different methods are used to answer this question such as, Neural Networks [3] , gradients computation [23], or probability. Gaze is often used as an a priori to perform an intended task (e.g, our work with ProMPs) to detect the object of interest (e.g., [12] with Neural Networks) , or to predict the goal location (e.g., [22] with dynamic models). The main differences between our study and [12,22] is that these works are interested in the human motion prediction while we associate human gaze to the robot motions.…”

Section: Related Workmentioning

confidence: 99%

Multi-modal Intention Prediction with Probabilistic Movement Primitives

Dermy

Charpillet

Ivaldi

2018

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

This paper proposes a method for multi-modal prediction of intention based on a probabilistic description of movement primitives and goals. We target dyadic interaction between a human and a robot in a collaborative scenario. The robot acquires multi-modal models of collaborative action primitives containing gaze cues from the human partner and kinetic information about the manipulation primitives of its arm. We show that if the partner guides the robot with the gaze cue, the robot recognizes the intended action primitive even in the case of ambiguous actions. Furthermore, this prior knowledge acquired by gaze greatly improves the prediction of the future intended trajectory during a physical interaction. Results with the humanoid iCub are presented and discussed.

show abstract

Human intention inference through interacting multiple model filtering

Cited by 20 publications

References 21 publications

Detecting errors in pick and place procedures

Detecting errors in pick and place procedures

Designing Robot Behavior in Human-Robot Interactions

Multi-modal Intention Prediction with Probabilistic Movement Primitives

Contact Info

Product

Resources

About