Toward Personalized Emotion Recognition: A Face Recognition Based Attention Method for Facial Emotion Recognition

Shahabinejad, Mostafa; Wang, Yang; Yu, Yuanhao; Tang, Jin Peng; Li, Jiani

doi:10.1109/fg52635.2021.9666982

Cited by 12 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By incorporating eyewitness descriptions with automated algorithms, these frameworks produce more reliable forensic sketches. In parallel, the focus has shifted towards improving forensic face sketch recognition [5]. Feature matching techniques compare distinctive features extracted from sketches with those from real-life facial images [18].…”

Section: Related Workmentioning

confidence: 99%

Forensic face-sketch creation and recognition using AWS Rekognition and facenet

Ali,

et al. 2024

AJCT

View full text Add to dashboard Cite

Forensic science faces significant challenges due to the time-consuming nature of hand-drawn face sketches, which hinder prompt criminal identification. Traditional methods, relying on forensic artists, are not only resourceintensive but also plagued by delays. The integration of modern technologies, such as deep learning and cloud infrastructure, presents a complex challenge, particularly concerning the interpretation of diverse user inputs and ensuring real-time compatibility with police databases while addressing data privacy concerns. To address these challenges effectively, this paper proposes a comprehensive solution. The focal point is on robust algorithmic development, a user-friendly design, and a secure cloud infrastructure. The envisioned standalone application aims to revolutionize the face sketch creation process, enabling users to effortlessly generate composite sketches through a drag-anddrop interface. By leveraging advanced deep learning techniques, the application seeks to bridge the gap between intuitive sketch creation and efficient facial recognition, thereby streamlining the entire investigative process.Sketch artists are utilized in criminal investigations to create facial composites based on eyewitness descriptions. However, this method is time-consuming, subjective, and reliant on the availability of skilled artists, posing challenges in accuracy and resource allocation for law enforcement agencies. As technology advances, there is a growing need to explore more efficient and objective alternatives to traditional hand-drawn sketches.

show abstract

Section: Related Workmentioning

confidence: 99%

Forensic face-sketch creation and recognition using AWS Rekognition and facenet

Ali,

et al. 2024

AJCT

View full text Add to dashboard Cite

show abstract

“…An example approach for individual-level personalized expression recognition include supervised domain adaption with mixture of experts [10]. Another approach by Shahabinejad et al [40] proposed a CNN architecture to learn and propagate individual deep facial features followed by a spatial attention map, which is then provided as an input to another CNN. For the task of personality computing, Shao et al [41] proposed to learn individual-specifc graph representations for personality traits recognition in a human-human interaction scenario.…”

Section: Personalized Models In Human-machine Interactionmentioning

confidence: 99%

“…Until recently, the Human-Machine Interaction (HMI) community has focused on one-fts-all approaches. Few approaches have attempted to train personalized models for afective and personality computing tasks such as mood [42], engagement [35], and emotion recognition [40], or personality traits prediction [39]. Among the employed machine learning methods for learning personalized models for HMI tasks, multitask learning was used to learn individual-specifc models for mood and stress prediction.…”

Section: Introductionmentioning

confidence: 99%

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Raghuram

Salam

Nasir

et al. 2022

Proceedings of the 2022 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defned as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identifed which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose:(1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profle are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fxed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the * Both authors contributed equally to this paper.

show abstract

“…Prior research in transfer learning for facial analysis tasks mostly focuses on transfer learning for the same task in order to bridge domain gaps such as personalization of a prediction system to specific individuals [8], [9], [10], [11], [12], [13], improving results on a benchmark by fine-tuning neural networks that are pre-trained on external datasets for a similar prediction task [14], or improving results by pre-training on a related facial analysis task [15], [16]. In contrast, our work tackles the more challenging transfer learning across domains and tasks, which is a form of transductive transfer learning [17].…”

Section: Introductionmentioning

confidence: 99%

ATL-BP: A Student Engagement Dataset and Model for Affect Transfer Learning for Behavior Prediction

Ruiz

Allessio

et al. 2023

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

We propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS) by analyzing their faces and gestures. The ability to predict such outcomes enables tutoring systems to adjust interventions and ultimately yield improved student learning. We collected and released a labeled dataset of 2,749 problem-solving interaction samples of 54 students working with an intelligent online math tutor. Our transfer-learning challenge was then to design a representation in the source domain of images obtained from the Internet for facial expression analysis, and transfer this learned representation for human behavior prediction in the domain of webcam videos of students in a classroom environment. We developed a novel facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We designed several variants of a recurrent neural network that models the temporal structure of video sequences. Our final model, named ATL-BP for Affect Transfer Learning for Behavior Prediction, achieves a relative increase in the mean F-score of 50% over the state-of-the-art method on this new dataset. We also propose an additional set of annotations to predict students' engagement while solving a specific problem, and present models that can predict such engagement.

show abstract

Toward Personalized Emotion Recognition: A Face Recognition Based Attention Method for Facial Emotion Recognition

Cited by 12 publications

References 27 publications

Forensic face-sketch creation and recognition using AWS Rekognition and facenet

Forensic face-sketch creation and recognition using AWS Rekognition and facenet

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

ATL-BP: A Student Engagement Dataset and Model for Affect Transfer Learning for Behavior Prediction

Contact Info

Product

Resources

About