An extensible architecture for robust multimodal human-robot communication

Rossi, Silvia; Leone, Enrico; Fiore, Michelangelo; Finzi, Alberto; Cutugno, Francesco

doi:10.1109/iros.2013.6696665

Cited by 35 publications

(16 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HMM uses both observation probabilities (absolute probability p(x)) and transitions abilities (conditional probability p(Y/X)) for modeling associations P(x, y) among NLC-related knowledge [76] [64]. With HMM models, tackled problems mainly include real-time task assignments [77], dynamic humancentered cooperation adjustment [64] [78], accurate tool delivering by simultaneously fusing multi-view data such as NL instruction, shoulder coordinates, shoulderselbows' 3D angle data, and hand poses [76] [79]. Limited by Markov assumptions, HMM is only capable of modeling shallow-level hidden correlations among NLCrelated knowledge.…”

Section: A Modelsmentioning

confidence: 99%

A review of methodologies for natural-language-facilitated human–robot cooperation

Liu

Zhang

2019

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Natural-language-facilitated human-robot cooperation (NLC) refers to using natural language (NL) to facilitate interactive information sharing and task executions with a common goal constraint between robots and humans. Recently, NLC research has received increasing attention. Typical NLC scenarios include robotic daily assistance, robotic health caregiving, intelligent manufacturing, autonomous navigation, and robot social accompany. However, a thorough review, that can reveal latest methodologies to use NL to facilitate human-robot cooperation, is missing. In this review, a comprehensive summary about methodologies for NLC is presented. NLC research includes three main research focuses: NL instruction understanding, NLbased execution plan generation, and knowledgeworld mapping. In-depth analyses on theoretical methods, applications, and model advantages and disadvantages are made. Based on our paper review and perspective, potential research directions of NLC are summarized. Index Terms-natural language, human-robot cooperation, NL instruction understanding, NLbased execution plan generation, knowledge-world mapping _______________________________________ Rui Liu and Xiaoli Zhang are with the Intelligent Robotics and Systems Lab (IRSL), Colorado School of Mines,

show abstract

Section: A Modelsmentioning

confidence: 99%

A review of methodologies for natural-language-facilitated human–robot cooperation

Liu

Zhang

2019

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…Rossi et al 24 addressed a multichannel fusion problem aimed for a robust communication between human and robot. In the case study that they presented, they described a generic and extensible architecture that included gesture and voice recognition, plus a fusion engine to improve the robustness of the interpretation of the human-robot communication.…”

Section: Related Workmentioning

confidence: 99%

Natural multimodal communication for human–robot collaboration

Maurtua

Fernández

Tellaeche

et al. 2017

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

This article presents a semantic approach for multimodal interaction between humans and industrial robots to enhance the dependability and naturalness of the collaboration between them in real industrial settings. The fusion of several interaction mechanisms is particularly relevant in industrial applications in which adverse environmental conditions might affect the performance of vision-based interaction (e.g. poor or changing lighting) or voice-based interaction (e.g. environmental noise). Our approach relies on the recognition of speech and gestures for the processing of requests, dealing with information that can potentially be contradictory or complementary. For disambiguation, it uses semantic technologies that describe the robot characteristics and capabilities as well as the context of the scenario. Although the proposed approach is generic and applicable in different scenarios, this article explains in detail how it has been implemented in two real industrial cases in which a robot and a worker collaborate in assembly and deburring operations.

show abstract

“…This theme is addressed in [1] where a multimodal architecture is used for fusing and interpreting the input from different sources, like voice and gestures.…”

Section: Related Workmentioning

confidence: 99%

A Bayesian approach for task recognition and future human activity prediction

Magnanimo

Saveriano

Rossi

et al. 2014

The 23rd IEEE International Symposium on Robot and Human Interactive Communication

Self Cite

View full text Add to dashboard Cite

Task recognition and future human activity prediction are of importance for a safe and profitable humanrobot cooperation. In real scenarios, the robot has to extract this information merging the knowledge of the task with contextual information from the sensors, minimizing possible misunderstandings. In this paper, we focus on tasks that can be represented as a sequence of manipulated objects and performed actions. The task is modelled with a Dynamic Bayesian Network (DBN), which takes as input manipulated objects and performed actions. Objects and actions are separately classified starting from RGB-D raw data. The DBN is responsible for estimating the current task, predicting the most probable future pairs of action-object and correcting possible misclassification. The effectiveness of the proposed approach is validated on a case of study, consisting of three typical tasks of a kitchen scenario.

show abstract

An extensible architecture for robust multimodal human-robot communication

Cited by 35 publications

References 16 publications

A review of methodologies for natural-language-facilitated human–robot cooperation

A review of methodologies for natural-language-facilitated human–robot cooperation

Natural multimodal communication for human–robot collaboration

A Bayesian approach for task recognition and future human activity prediction

Contact Info

Product

Resources

About