Multimodal Gesture Recognition for Mascot Robot System Based on Choquet Integral Using Camera and 3D Accelerometers Fusion

Tang, Yuxiang; Vu, Hai; Le, Phuc Q.; Masano, D.; Thet, O.; Fatichah, Chastine; Liu, Zhentao; Yamaguchi, Masashi; Tangel, Martin Leonard; Dong, Fangyan; Yamazaki, Youichi; Hirota, Kaoru

doi:10.20965/jaciii.2011.p0563

Cited by 8 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How to convey information is important for AI and robots that support our daily lives. The importance of casual verbal and non-verbal interactions with robots at home has been pointed out [16][17][18] environments potentially will be used to observe elderly people at home [6].…”

Section: Recreation With Robots At Homementioning

confidence: 99%

Frailty Care Robot for Elderly and its Application for Physical and Psychological Support

Yamazaki

Ishii²,

Ito³

et al. 2021

JACIII

Self Cite

View full text Add to dashboard Cite

To achieve continuous frail care in the daily lives of the elderly, we propose AHOBO, a frail care robot for the elderly at home. Two types of support systems by AHOBO were implemented to support the elderly in both physical health and psychological aspects. For physical health frailty care, we focused on blood pressure and developed a support system for blood pressure measurement with AHOBO. For psychological frailty care, we implemented reminiscent coloring with the AHOBO as a recreational activity with the robot. The usability of the system was evaluated based on the assumption of continuous use in daily life. For the support system in blood pressure measurement, we performed a qualitative evaluation using a questionnaire for 16 subjects, including elderly people under blood pressure measurement by the system. The results confirmed that the proposed robot does not affect the blood pressure readings and is acceptable in terms of ease of use based on subjective evaluation. For the reminiscent coloring interaction, subjective evaluation was conducted on two elderly people under the verbal fluency task, and it has been confirmed that the interaction can be used continuously in daily life. The widespread use of the proposed robot as an interface for AI that supports daily life will lead to a society in which AI robots support people from the cradle to the grave.

show abstract

Section: Recreation With Robots At Homementioning

confidence: 99%

Frailty Care Robot for Elderly and its Application for Physical and Psychological Support

Yamazaki

Ishii²,

Ito³

et al. 2021

JACIII

Self Cite

View full text Add to dashboard Cite

show abstract

“…Atmosphere is estimated based on the emotion state of the agents. Gestures/postures is recognized by gesture recognition algorithms [6] from sensors like cameras and accelerometers. The speech contents, universal and customized knowledge, emotions, atmospheres, and gestures/postures are important input for the situation inference system.…”

Section: Inference Framework For Deep Level Situation Understandingmentioning

confidence: 99%

“…Most works on gesture recognition for Human-Robot Interaction (HRI) have been done based on visual information, such as sign tracking and recognition system (STARS) [5]. To improve the robustness of gesture recognition system, a Choquet integral based multimodal gesture recognition system [6] is proposed. Emotions and intentions are also important in HRI.…”

Section: Introductionmentioning

confidence: 99%

Deep Level Situation Understanding for Casual Communication in Humans-Robots Interaction

Tang

Dong

Yamazaki

et al. 2015

Int. J. Fuzzy Log. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

A concept of Deep Level Situation Understanding is proposed to realize human-like natural communication (called casual communication) among multi-agent (e.g., humans and robots/machines), where the deep level situation understanding consists of surface level understanding (such as gesture/posture understanding, facial expression understanding, speech/voice understanding), emotion understanding, intention understanding, and atmosphere understanding by applying customized knowledge of each agent and by taking considerations of thoughtfulness. The proposal aims to reduce burden of humans in humans-robots interaction, so as to realize harmonious communication by excluding unnecessary troubles or misunderstandings among agents, and finally helps to create a peaceful, happy, and prosperous humans-robots society. A simulated experiment is carried out to validate the deep level situation understanding system on a scenario where meeting-room reservation is done between a human employee and a secretary-robot. The proposed deep level situation understanding system aims to be applied in service robot systems for smoothing the communication and avoiding misunderstanding among agents.

show abstract

“…It is a two-layer network system consisted of information sensing and transfer layer as well as data processing layer. Each robot can be regarded as a net- There are eight modules in the RTM-Network, including the speech recognition module (SRM), gesture recognition module (GRM), eyerobot control module (ERCM), emotion processing module (EPM), display module, mobile robot control module, scenario server module, and information retrieval module [32].…”

Section: Evaluation Of Graphic Visualization For Atmosphere Representmentioning

confidence: 99%

Multi-robot behavior adaptation to local and global communication atmosphere in humans-robots interaction

Chen

Liu

et al. 2014

J Multimodal User Interfaces

Self Cite

View full text Add to dashboard Cite

Multi-robot behavior adaptation mechanism based on cooperative-neutral-competitive fuzzy-Q learning is proposed for coordinating local communication atmospheres in humans-robots interaction, in which the communication atmosphere is represented by a two-layer fuzzy fusion model and is visualized by shape-color-fill-wave graphics. It aims to realize smooth communication between humans and robots in the local and global communication atmosphere coexistent interaction by decreasing the response time of robots and social distance between humans and robots, as well as visualizing the communication atmosphere. Experiments on multi-robot behavior adaptation are performed in a virtual home party environment. Results show that the proposal saves 47 and 103 learning steps (i.e., the learning rate is increased by 72 % and 85 %) compared to fuzzy production rule based friend-Q learning (FPRFQ) and friend-Q learning (FQ), respectively; the distance between humangenerated atmosphere and robot-generated atmosphere is 3 times and 7 times shorter than the FPRFQ and the FQ, respectively. Additionally, subjective estimation of graphic visualization of the atmosphere through questionnaire obtains 85.5 % accuracy for shape, 77.2 % for color, 65.3 % for fill, and 91.7 % for wave. The proposed mechanism is being extended to the robot behavior adaptation to international communication atmosphere, where the atmosphere is generated by people from different countries with different cultural backgrounds.

show abstract

Multimodal Gesture Recognition for Mascot Robot System Based on Choquet Integral Using Camera and 3D Accelerometers Fusion

Cited by 8 publications

References 12 publications

Frailty Care Robot for Elderly and its Application for Physical and Psychological Support

Frailty Care Robot for Elderly and its Application for Physical and Psychological Support

Deep Level Situation Understanding for Casual Communication in Humans-Robots Interaction

Multi-robot behavior adaptation to local and global communication atmosphere in humans-robots interaction

Contact Info

Product

Resources

About