Real-Time Recognition of Facial Expressions Using Facial Electromyograms Recorded Around the Eyes for Social Virtual Reality Applications

Cha, Ho-Seung; Choi, Seo-Youn; Im, Chang‐Hwan

doi:10.1109/access.2020.2983608

Cited by 31 publications

(22 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using VR technology to implement collaborative virtual environments (CVEs) allows for co-presence [93] and communication using not only speech and potentially video, but also gestures, posture [127], and even facial expressions [19] for geographically dislocated users. All these communication channels are available naturally in real world co-located collaboration, but have to be artificially enabled in a CVE.…”

Section: C11: Overcoming Constraints Of Realitymentioning

confidence: 99%

Grand Challenges in Immersive Analytics

Ens

Bach

Cordeil

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

147

View full text Add to dashboard Cite

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and humancomputer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

show abstract

Section: C11: Overcoming Constraints Of Realitymentioning

confidence: 99%

Grand Challenges in Immersive Analytics

Ens

Bach

Cordeil

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

147

View full text Add to dashboard Cite

show abstract

“…In recent work, [38] proposed a FER system based on Facial Electromyograms (fEMG); such system was based on Riemannian approach to reduce the number of training data sets needed and improve the performance of FER. Furthermore, they developed an online FER system in order to perform face recognition of virtual social reality users in real time.…”

Section: B Facial Expression Recognition Toolsmentioning

confidence: 99%

WHOT, a Novel Tool to Assist Women Victims of Violence: A Case Study in the Brazilian Amazon

et al. 2021

View full text Add to dashboard Cite

Violence against women is a problem faced in several ways, in various societies; however, the introduction of computational tools is something still little explored in this confrontation. Thus, it is necessary to invest in researches that bring technological development closer to the prevention, discovery, and combat of this form of violence. This paper presents the Women's Health Observer Tool (WHOT) that helps to build psychobehavioral profiles of women victims of violence, based on three features: i) recognition of facial expressions to infer emotions; ii) provision of digital questionnaires on intimate partner violence (IPV), adverse childhood experiences (ACE) and post-traumatic stress disorder (PTSD); and iii) generation of individual reports with cross-references of statistical analysis between the data obtained in each interview. To validate the tool, a case study was conducted with 50 women assisted in basic health units in a city of the Brazilian Amazon for prenatal care. The results are satisfactory for the use of the tool, which was able to infer emotions (joy, surprise, sadness, and anger), and the prevalence of sadness (25.24%) was verified among the interviewees. For ACE, the majority (21) of the women reported having suffered only physical abuse; as for IPV, the majority of the interviewees (27) reported no abuse; and 78% of the women (39) had no indicative signs of PTSD. The results further point out that there is 3.94 more chance that the group of women who reported any abuse, either in childhood or adulthood, compared to the reference group, would develop PTSD.

show abstract

“…To address the above issues, facial electromyogram (fEMG) has been recorded around the eyes to recognize facial expressions (Yang and Yang 2011;Fatoorechi et al 2017;Hamedi et al 2018;Phinyomark and Scheme 2018;Lou et al 2020;Cha et al 2020). An fEMG indicates the electrical activity generated by facial muscle movements, which can be easily recorded using electrodes attached to the face.…”

Section: Introductionmentioning

confidence: 99%

“…Over the past decades, various fEMG-based FER systems have been proposed as shown in Table 1 (Hamedi et al 2011(Hamedi et al , 2013(Hamedi et al , 2018Rezazadeh et al 2011;Cha et al 2020). It is to be noted that electrode locations reported in these studies were not determined considering VR applications; therefore, electrode locations varied from study to study.…”

Section: Introductionmentioning

confidence: 99%

“…However, this system required users to make facial expression four times for the registration, which does not seem to be practical enough to be used in real VR environments considering that the current FER systems require the users to repeat the registration process whenever they use the system. To address this issue, we suggested a new fEMG-based FER system in which only a single trial for each facial expression was necessary to build the classification model (Cha et al 2020). We also implemented a real-time FER system with a processing time less than 0.05 s and succeeded in reflecting user's current facial expressions onto a virtual avatar's face in real time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance enhancement of facial electromyogram-based facial-expression recognition for social virtual reality applications using linear discriminant analysis adaptation

Cha

2021

Virtual Reality

Self Cite

View full text Add to dashboard Cite

Recent studies have indicated that facial electromyogram (fEMG)-based facial-expression recognition (FER) systems are promising alternatives to the conventional camera-based FER systems for virtual reality (VR) environments because they are economical, do not depend on the ambient lighting, and can be readily incorporated into existing VR headsets. In our previous study, we applied a Riemannian manifold-based feature extraction approach to fEMG signals recorded around the eyes and demonstrated that 11 facial expressions could be classified with a high accuracy of 85.01%, with only a single training session. However, the performance of the conventional fEMG-based FER system was not high enough to be applied in practical scenarios. In this study, we developed a new method for improving the FER performance by employing linear discriminant analysis (LDA) adaptation with labeled datasets of other users. Our results indicated that the mean classification accuracy could be increased to 89.40% by using the LDA adaptation method (p < .001, Wilcoxon signed-rank test). Additionally, we demonstrated the potential of a user-independent FER system that could classify 11 facial expressions with a classification accuracy of 82.02% without any training sessions. To the best of our knowledge, this was the first study in which the LDA adaptation approach was employed in a cross-subject manner. It is expected that the proposed LDA adaptation approach would be used as an important method to increase the usability of fEMG-based FER systems for social VR applications.

show abstract

Real-Time Recognition of Facial Expressions Using Facial Electromyograms Recorded Around the Eyes for Social Virtual Reality Applications

Cited by 31 publications

References 40 publications

Grand Challenges in Immersive Analytics

Grand Challenges in Immersive Analytics

WHOT, a Novel Tool to Assist Women Victims of Violence: A Case Study in the Brazilian Amazon

Performance enhancement of facial electromyogram-based facial-expression recognition for social virtual reality applications using linear discriminant analysis adaptation

Contact Info

Product

Resources

About