Behavioral Analysis of Kinetic Telepresence for Small Symmetric Group-to-Group Meetings

Otsuka, Kazuhiro

doi:10.1109/tmm.2017.2771396

Cited by 12 publications

(7 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Not only directly evaluating the ECA's head movements but also analyzing users' reactions in terms of their head movements would be an interesting topic. Furthermore, developing and evaluating telepresence systems are both within our scope [18]. Comparing face-to-face and telepresence settings in terms of emergent head-movement functions would indicate the quality of a telepresence system.…”

Section: Discussionmentioning

confidence: 99%

“…Adams et al found that head movements carry emotional information complementary to the facial expression [90] and can be useful input for the automatic recognition of emotional categories [91]. Otsuka suggested that head movements expressed with a robotic telepresence avatar increase the observer's perception of emotion compared with a static display showing the same face image without physical movements [18].…”

Section: ) Affect Displaymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Another application area involving head movements is computer-mediated communications (CMCs), especially robotic telepresence [18], which exploits robotic heads as surrogates of remote users. The robotic head, which typically represents or displays the remote user's face, is controlled automatically by tracking the user's head [18], [19] and/or controlled manually through user interfaces [19]. Intelligent control of robotic heads is an important research topic that seeks to compensate for communication deficits caused by transmission delays and limited audio/visual channels and to improve the interactivity among spatially separated people so that they can interact as if they were all together in the same place; here, the aim is to adaptively regulate the robot's movements depending on the user's intentions, as recognized from his/her head movements and other behaviors, and on the communication environment.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Analyzing Multifunctionality of Head Movements in Face-to-Face Conversations Using Deep Convolutional Neural Networks

Otsuka

Tsumori

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

A functional head-movement corpus and convolutional neural networks (CNNs) for detecting head-movement functions are presented for analyzing the multiple communicative functions of head movements in multiparty face-to-face conversations. First, focusing on the multifunctionality of head movements, i.e., that a single head movement can simultaneously perform multiple functions, this paper defines 32 nonmutually-exclusive function categories, whose genres are speech production, eliciting and giving feedback, turn management, and cognitive and affect display. To represent and capture arbitrary multifunctional structures, our corpus employs multiple binary codes and logical-sum-based aggregations of multiple coders' judgments. A corpus analysis targeting four-party Japanese conversations revealed multifunctional patterns in which the speaker modulates multiple functions, such as emphasis and eliciting listeners' responses, through rhythmic head movements, and listeners express various attitudes and responses through continuous back-channel head movements. This paper proposes CNN-based binary classifiers for detecting each of the functions from the angular velocity of the head pose and the presence or absence of utterances. The experimental results showed that the recognition performance varies greatly, from approximately 30% to 90% in terms of the F-score, depending on the function category, and the performance was positively correlated with the amount of data and inter-coder agreement. In addition, we noted a tendency toward overdetection that added more functions to those originally in the corpus. The analyses and experiments confirm that our approach is promising for studying the multifunctionality of head movements. INDEX TERMS deep neural networks, gesture recognition, meeting analysis, multimodal sensor, nonverbal behaviors, social signal processing This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: ) Affect Displaymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Analyzing Multifunctionality of Head Movements in Face-to-Face Conversations Using Deep Convolutional Neural Networks

Otsuka

Tsumori

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, it is preferable to place the camera in the line of sight of the observer. MMSpace [14] enables eye contact by installing cameras behind a transparent screen that displays the avatar. This system uses multiple cameras for one user and switches cameras based on the gaze direction of the user.…”

Section: B Telepresence Systemmentioning

confidence: 99%

Aerial Imaging by Retro-Reflection for Mid-Air Image Display and Camera Viewpoint Transfer

Tsuchiya

Koizumi

2021

IEEE Access

View full text Add to dashboard Cite

Previously, a telepresence system, called Levitar, was proposed to allow communication through computer-graphics (CG) avatars in real space. Levitar employs micro-mirror array plates (MMAPs) to display mid-air images and optically transfer the camera viewpoint to the mid-air image-display position optically. This system captures the first-person view of a mid-air image avatar; however, it has a limitation that the stray light caused by MMAPs reduces its range. Therefore, in this study, we propose an optical system to capture a wider range of mid-air viewpoints. The proposed optical system uses aerial imaging by retro-reflection (AIRR) instead of MMAPs for mid-air image display and camera viewpoint transfer. AIRR is a mid-air imaging system that includes a beam splitter and retro-reflector. However, in contrast to MMAPs, it does not generate stray light. The optical transfer of the camera viewpoint by AIRR enables the capture of a large area. To evaluate the range of the capturing area when using cameras with MMAPs and AIRR, we conducted experiments under different camera gaze directions and effective angle of view. The results indicate that AIRR maintains a constant resolution up to 40 • of the camera gaze direction, whereas MMAPs have a lower resolution above 30 • , and the area of the marker detection rate of over 80% is approximately 54% larger for AIRR than for MMAPs. This implies that AIRR achieved a wider imaging range than MMAPs for mid-air telepresence systems.INDEX TERMS Avatar, mid-air image, viewpoint transfer, wide imaging range. KEI TSUCHIYA received the B.E. degree from the University of Electro-Communications, Japan, in 2019. His research interests include mid-air imaging and human-computer interactions.

show abstract

Language-Driven Ordinal Learning for Imbalanced Head Pose Estimation

Wang,

Yu,

Lin

et al. 2024

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Behavioral Analysis of Kinetic Telepresence for Small Symmetric Group-to-Group Meetings

Cited by 12 publications

References 53 publications

Analyzing Multifunctionality of Head Movements in Face-to-Face Conversations Using Deep Convolutional Neural Networks

Analyzing Multifunctionality of Head Movements in Face-to-Face Conversations Using Deep Convolutional Neural Networks

Aerial Imaging by Retro-Reflection for Mid-Air Image Display and Camera Viewpoint Transfer

Language-Driven Ordinal Learning for Imbalanced Head Pose Estimation

Contact Info

Product

Resources

About