Performance enhancement of facial electromyogram-based facial-expression recognition for social virtual reality applications using linear discriminant analysis adaptation

Abstract

Recent studies have indicated that facial electromyogram (fEMG)-based facial-expression recognition (FER) systems are promising alternatives to the conventional camera-based FER systems for virtual reality (VR) environments because they are economical, do not depend on the ambient lighting, and can be readily incorporated into existing VR headsets. In our previous study, we applied a Riemannian manifold-based feature extraction approach to fEMG signals recorded around the eyes and demonstrated that 11 facial expressions could be classified with a high accuracy of 85.01%, with only a single training session. However, the performance of the conventional fEMG-based FER system was not high enough to be applied in practical scenarios. In this study, we developed a new method for improving the FER performance by employing linear discriminant analysis (LDA) adaptation with labeled datasets of other users. Our results indicated that the mean classification accuracy could be increased to 89.40% by using the LDA adaptation method (p < .001, Wilcoxon signed-rank test). Additionally, we demonstrated the potential of a user-independent FER system that could classify 11 facial expressions with a classification accuracy of 82.02% without any training sessions. To the best of our knowledge, this was the first study in which the LDA adaptation approach was employed in a cross-subject manner. It is expected that the proposed LDA adaptation approach would be used as an important method to increase the usability of fEMG-based FER systems for social VR applications.

Blockchain for the Metaverse: A Review

“…Some of the sensitive data from the users such as bank/credit card details are acquired when the users make payments. In addition, sensitive data such as biometrics poses/gestures of the users have to be acquired in the metaverse to create digital avatars [32]. Data acquisition helps in training the AI/ML algorithms that can assist in decision making, digital product development, recommendation system development, and marketing in the metaverse [15].…”

Seeing the faces is so important—Experiences from online team meetings on commercial virtual reality platforms

“…Considering the study by Abdullah et al (2021), this could also explain the participants' decreased activity to maintain social connectedness and less person-directed gaze in VR conversations compared to videoconferencing. With rapid progress in research on the recognition of facial expressions (Hamedi et al, 2018;Lou et al, 2020;Cha and Im, 2022) and eye movement (Schwartz et al, 2020), as well as their mapping to the avatar, more expressive face and gaze representation will be available in the near future. We expect this to be a crucial facilitator for authentic social exchange in VR.…”

Review of Virtual Reality (VR) Applications To Enhance Chemical Safety: From Students to Plant Operators

“…Therefore, facial electromyography (fEMG), a physiological sensor that is more sensitive to emotions than video-based facial expression analytics, can be used instead. 102 McGhee et al demonstrated that fEMG could accurately distinguish facial expressions and emotional reactions in the virtual world. 103 In summary, measurements from various physiological sensors can be used to understand human cognitive behavior and help in keeping track of the learning progress.…”

“…However, in a VR system, the face of the user may be obstructed by a head-mounted VR device, thus limiting the webcam recording of the user’s emotions. Therefore, facial electromyography (fEMG), a physiological sensor that is more sensitive to emotions than video-based facial expression analytics, can be used instead . McGhee et al demonstrated that fEMG could accurately distinguish facial expressions and emotional reactions in the virtual world…”