Mental states and personality based on real-time physical activity and facial expression recognition

Abstract

IntroductionTo explore a quick and non-invasive way to measure individual psychological states, this study developed interview-based scales, and multi-modal information was collected from 172 participants.MethodsWe developed the Interview Psychological Symptom Inventory (IPSI) which eventually retained 53 items with nine main factors. All of them performed well in terms of reliability and validity. We used optimized convolutional neural networks and original detection algorithms for the recognition of individual facial expressions and physical activity based on Russell's circumplex model and the five factor model.ResultsWe found that there was a significant correlation between the developed scale and the participants' scores on each factor in the Symptom Checklist-90 (SCL-90) and Big Five Inventory (BFI-2) [r = (−0.257, 0.632), p < 0.01]. Among the multi-modal data, the arousal of facial expressions was significantly correlated with the interval of validity (p < 0.01), valence was significantly correlated with IPSI and SCL-90, and physical activity was significantly correlated with gender, age, and factors of the scales.DiscussionOur research demonstrates that mental health can be monitored and assessed remotely by collecting and analyzing multimodal data from individuals captured by digital tools.

Application of EEG-based Machine Learning in Stock Trading-related Emotion Recognition

“…The feature extraction procedure involves acquiring information concerning the electrode locations and their attributes within the frequency domain. The EEG data is spatially referenced to digitally linked ears (DLE) [17], providing spatial insights into the source locations for each electrode.…”

Enhancing the conversational agent with an emotional support system for mental health digital therapeutics

“…An effective ES system should consider all mental states of the whole conversation. Identifying the user's fine-grained, dynamic mental state is critical in the multi-turn ESC scenario ( 15 , 33 ). Moreover, some earlier works merely considered the dialog history to foresee the strategy and overlooked the past strategies the supporter used.…”

Summarize: Study Examines Effect of Psychological Nursing Care on Outpatients

“…1. Observing the facial expressions [6] and body language [7] of waiting patients helps assess their psychological state and provide real-time attention to reduce their anxiety and improve their compliance with medical treatment;…”