Andorid Robot Motion Generation Based on Video-Recorded Human Demonstrations

Go, Dae-Seong; Hyung, Hyun-Jun; Lee, Dong‐Wook; Yoon, Han Ul

doi:10.1109/roman.2018.8525727

Cited by 4 publications

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Synthesizing co‐speech gestures is essential for creating interactive and believable virtual characters in human‐computer interaction (HCI), graphics and social robotics. Thus significant effort has been applied and progress has been made for applications in virtual agents [CVB01, PB03, KW04, NKAS08, CMM15] and humanoid robots [SKWJ09, YKJ*19, LHP11, GHLY18, NTHLO10, HS16]. Neff identifies the two main sub‐problems of generating gesture as the specification problem and the animation problem [Nef16].…”

Section: Approaches For Gesture Synthesismentioning

confidence: 99%

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

Nyatsanga

Kucherenko²,

Ahuja³

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co‐speech gestures is a long‐standing problem in computer animation and is considered an enabling technology for creating believable characters in film, games, and virtual social spaces, as well as for interaction with social robots. The problem is made challenging by the idiosyncratic and non‐periodic nature of human co‐speech gesture motion, and by the great diversity of communicative functions that gestures encompass. The field of gesture generation has seen surging interest in the last few years, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep‐learning‐based generative models that benefit from the growing availability of data. This review article summarizes co‐speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule‐based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text and non‐linguistic input. Concurrent with the exposition of deep learning approaches, we chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method (e.g., optical motion capture or pose estimation from video). Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human‐like motion; grounding the gesture in the co‐occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.

show abstract

Section: Approaches For Gesture Synthesismentioning

confidence: 99%

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

Nyatsanga

Kucherenko²,

Ahuja³

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…The information for each frame is further divided into 3D model and avatar motion and other events. Various methods of estimating avatar posture exist for VR [7], [8], but dealing with every movement is difficult. There are also movements when avatars and objects collide.…”

Section: Recording Vr Activitiesmentioning

confidence: 99%

Virtual Reality Campuses as New Educational Metaverses

NAGAO

2023

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

This paper focuses on the potential value and future prospects of using virtual reality (VR) technology in online education. In detailing online education and the latest VR technology, we focus on metaverse construction and artificial intelligence (AI) for educational VR use. In particular, we describe a virtual university campus in which on-demand VR lectures are conducted in virtual lecture halls, automated evaluations of student learning and training using machine learning, and the linking of multiple digital campuses.

show abstract

“…The waist and neck have three degrees of freedom to allow EveR to rotate its waist and neck like humans. Finally, both arms can express a range of gestures by using 12 motors [13,14].…”

Section: Ever Androidsmentioning

confidence: 99%

“…Behavioral expression technology is currently aimed at enabling androids to exactly imitate human facial expressions and gestures [13][14][15]. The behavioral expressions of robots can largely be classified as facial expressions, gestures, lip syncing and eye contact [16,17], which also vary depending on if there are one or multiple interaction targets.…”

Section: Introductionmentioning

confidence: 99%

Optimizing Android Facial Expressions Using Genetic Algorithms

et al. 2019

Self Cite

View full text Add to dashboard Cite

Because the internal structure, degree of freedom, skin control position and range of the android face are different, it is very difficult to generate facial expressions by applying existing facial expression generation methods. In addition, facial expressions differ among robots because they are designed subjectively. To address these problems, we developed a system that can automatically generate robot facial expressions by combining an android, a recognizer capable of classifying facial expressions and a genetic algorithm. We have developed two types (older men and young women) of android face robots that can simulate human skin movements. We selected 16 control positions to generate the facial expressions of these robots. The expressions were generated by combining the displacements of 16 motors. A chromosome comprising 16 genes (motor displacements) was generated by applying real-coded genetic algorithms; subsequently, it was used to generate robot facial expressions. To determine the fitness of the generated facial expressions, expression intensity was evaluated through a facial expression recognizer. The proposed system was used to generate six facial expressions (angry, disgust, fear, happy, sad, surprised); the results confirmed that they were more appropriate than manually generated facial expressions.

show abstract

Andorid Robot Motion Generation Based on Video-Recorded Human Demonstrations

Cited by 4 publications

References 3 publications

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

Virtual Reality Campuses as New Educational Metaverses

Optimizing Android Facial Expressions Using Genetic Algorithms

Contact Info

Product

Resources

About