Emotion Recognition is attracting the attention of the research community due to the multiple areas where it can be applied, such as in healthcare or in road safety systems. In this paper, we propose a multimodal emotion recognition system that relies on speech and facial information. For the speech-based modality, we evaluated several transfer-learning techniques, more specifically, embedding extraction and Fine-Tuning. The best accuracy results were achieved when we fine-tuned the CNN-14 of the PANNs framework, confirming that the training was more robust when it did not start from scratch and the tasks were similar. Regarding the facial emotion recognizers, we propose a framework that consists of a pre-trained Spatial Transformer Network on saliency maps and facial images followed by a bi-LSTM with an attention mechanism. The error analysis reported that the frame-based systems could present some problems when they were used directly to solve a video-based task despite the domain adaptation, which opens a new line of research to discover new ways to correct this mismatch and take advantage of the embedded knowledge of these pre-trained models. Finally, from the combination of these two modalities with a late fusion strategy, we achieved 80.08% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. The results revealed that these modalities carry relevant information to detect users’ emotional state and their combination enables improvement of system performance.
Laboratory studies were conducted to determine leaching of Cu, Cr, and B from pine sapwood treated with Cr-Cu-B (CCB) preservative in different leaching media (ultrapure water [UPW], rainwater [RW], and soil water [SW]). Losses of metals from CCB-treated wood, expressed in percentage of initial content, were always highest for B (11.2-63.5%) and lowest for Cr (0.02-0.36%) in any of the leaching media assayed. Losses of B and Cr from treated wood were not affected by the leaching medium; however, losses of Cu were 1.5 to 2.5 times higher in SW than in UPW and RW. The influence of drying time between successive leaching periods on release of metals also was determined. Treated wood was submitted to wetting-drying cycles of 24 h/d, 18 h/d, and 6 h/d in contact with the leaching medium over a total contact time of 276 h. For an equivalent contact period of CCB-treated wood with either UPW or RW, leaching of B was from 1.1 to 1.9 times higher for continual wetting than under intermittent wetting conditions. Leaching of Cu was not affected by the wetting-drying periods when wood was in contact with RW, but it was 1.2 to 1.5 times higher for continual wetting with UPW. In general, leaching of Cr was not affected by the wetting-drying cycles when in contact with UPW or RW. However, when CCB-treated wood was in contact with SW under discontinuous wetting of 6 h/d, Cu and Cr emissions were approximately 1.2 and 2.5 times higher, respectively, than with continuous wetting. This study shows that the type of leaching medium and the drying time between successive wetting periods must be considered when evaluating metal losses from CCB-treated wood during service life.
Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated a pre-trained xlsr-Wav2Vec2.0 transformer using two transfer-learning techniques: embedding extraction and fine-tuning. The best accuracy results were achieved when we fine-tuned the whole model by appending a multilayer perceptron on top of it, confirming that the training was more robust when it did not start from scratch and the previous knowledge of the network was similar to the task to adapt. Regarding the facial emotion recognizer, we extracted the Action Units of the videos and compared the performance between employing static models against sequential models. Results showed that sequential models beat static models by a narrow difference. Error analysis reported that the visual systems could improve with a detector of high-emotional load frames, which opened a new line of research to discover new ways to learn from videos. Finally, combining these two modalities with a late fusion strategy, we achieved 86.70% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. Results demonstrated that these modalities carried relevant information to detect users’ emotional state and their combination allowed to improve the final system performance.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.