Multimodal emotion recognition has gained traction in affective computing research community to overcome the limitations posed by the processing a single form of data and to increase recognition robustness. In this study, a novel emotion recognition system is introduced, which is based on multiple modalities including facial expressions, galvanic skin response (GSR) and electroencephalogram (EEG). This method follows a hybrid fusion strategy and yields a maximum one-subject-out accuracy of 81.2% and a mean accuracy of 74.2% on our bespoke multimodal emotion dataset (LUMED-2) for 3 emotion classes: sad, neutral and happy. Similarly, our approach yields a maximum one-subject-out accuracy of 91.5% and a mean accuracy of 53.8% on the Database for Emotion Analysis using Physiological Signals (DEAP) for varying numbers of emotion classes, 4 in average, including angry, disgust, afraid, happy, neutral, sad and surprised. The presented model is particularly useful in determining the correct emotional state in the case of natural deceptive facial expressions. In terms of emotion recognition accuracy, this study is superior to, or on par with, the reference subject-independent multimodal emotion recognition studies introduced in the literature.

The purpose of this study is to explore the effects of an affective recommendation system on the developmental trajectories of prospective teachers' emotional patterns, integrated with a Simulated Virtual Classroom (SVC) platform called SimInClass. SVC exposes teachers to a range of student discourses in the form of unexpected stimuli. Fifteen prospective teachers participated in a study consisting of two practicum sessions in the SVC. Participants did not receive any affective recommendation after the first session but did receive it after the second session. Additional data were collected during both sessions in the SVC, including the physiological responses, such as electroencephalogram (EEG), galvanic skin response (GSR), and facial expressions. L metric and Lag sequential analysis were employed in determining teachers' transitional emotional patterns. The results showed that participants did not maintain disgust after receiving affective recommendations, although they maintained sadness. This result indicates that the given affective recommendation has an inherent effect on negative emotions that are felt less intensely. Different or longer‐term interventions may be needed for more intense and long‐lasting negative discrete emotions such as sadness. Also, participants transitioned to happiness and sadness instead of maintaining their neutral status after receiving an affective recommendation. This result demonstrates that affective recommendations encourage participants to use the cognitive reappraisal necessary for emotion regulation. When the participants' emotional patterns are examined on the basis of student discourse, the results are more complex and the emotional patterns differ according to the function of the discourse triggered by virtual students. What is already known about this topic Teachers experience different emotional states during teaching. Teachers' emotions affect their behaviour management, teaching process and student engagement. It is beneficial to increase opportunities for prospective teachers' classroom experience and provide them with sufficient guidance and advice during this difficult process. What this paper adds The affective recommendation system has intervened in the persistence of short‐term and low‐intensity negative emotions. The affective recommendation system enabled prospective teachers to try to reach optimal emotional conditions through cognitive reappraisal processes. When emotional patterns are examined in light of the types of student discourse, it is noted that happy prospective teachers maintain their emotions when confronted with unexpected stimuli. However, prospective teachers in a negative valance displayed a descending pattern of activation in response to an unexpected stimulus. Implications for practice and/or policy Teacher emotions need to be taken into account in teacher education programs. SVCs can be utilized as useful tools for teacher education programs. In subsequent studies, it is suggested to explore the stimulus‐based effects of t...