A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors

This paper focuses on the binary classification of the emotion of fear, based on the physiological data and subjective responses stored in the DEAP dataset. We performed a mapping between the discrete and dimensional emotional information considering the participants’ ratings and extracted a substantial set of 40 types of features from the physiological data, which represented the input to various machine learning algorithms—Decision Trees, k-Nearest Neighbors, Support Vector Machine and artificial networks—accompanied by dimensionality reduction, feature selection and the tuning of the most relevant hyperparameters, boosting classification accuracy. The methodology we approached included tackling different situations, such as resolving the problem of having an imbalanced dataset through data augmentation, reducing overfitting, computing various metrics in order to obtain the most reliable classification scores and applying the Local Interpretable Model-Agnostic Explanations method for interpretation and for explaining predictions in a human-understandable manner. The results show that fear can be predicted very well (accuracies ranging from 91.7% using Gradient Boosting Trees to 93.5% using dimensionality reduction and Support Vector Machine) by extracting the most relevant features from the physiological data and by searching for the best parameters which maximize the machine learning algorithms’ classification scores.

Section: Fear An Adaptive Emotional Responsementioning

confidence: 99%

Machine Learning Methods for Fear Classification Based on Physiological Features

Petrescu

Oprea

et al. 2021

“…There are various sensors that can be used in a smartphone: multiple location-based sensors (GPS, gyroscopes), accelerometers, audio sensors, Bluetooth radios, Wi-Fi antennas, and with the advancement of technology, many other sensors–such as pulse or blood pressure sensors. In the field of healthcare, such passive data collection is becoming the main solution for health monitoring in the elderly or in other special scenarios [ 17 , 18 , 19 ].…”

Section: Introductionmentioning

confidence: 99%

Attitudes towards Participation in a Passive Data Collection Experiment

Ságvári

Gulyás

Koltai

2021

In this paper, we present the results of an exploratory study conducted in Hungary using a factorial design-based online survey to explore the willingness to participate in a future research project based on active and passive data collection via smartphones. Recently, the improvement of smart devices has enabled the collection of behavioural data on a previously unimaginable scale. However, the willingness to share this data is a key issue for the social sciences and often proves to be the biggest obstacle to conducting research. In this paper we use vignettes to test different (hypothetical) study settings that involve sensor data collection but differ in the organizer of the research, the purpose of the study and the type of collected data, the duration of data sharing, the number of incentives and the ability to suspend and review the collection of data. Besides the demographic profile of respondents, we also include behavioural and attitudinal variables to the models. Our results show that the content and context of the data collection significantly changes people’s willingness to participate, however their basic demographic characteristics (apart from age) and general level of trust seem to have no significant effect. This study is a first step in a larger project that involves the development of a complex smartphone-based research tool for hybrid (active and passive) data collection. The results presented in this paper help improve our experimental design to encourage participation by minimizing data sharing concerns and maximizing user participation and motivation.

“…For example, an intelligent HCI system that combines AI with wearable devices (e.g., data gloves) can solve communication problems between a hard-of-hearing and a non-disabled person [40]. Mobile IUI solutions today make use of the plethora of advanced sensors available in smartphones, such as camera, microphone, keyboard, touchscreen, depth sensors, accelerometer, gyroscope, geolocation sensor, barometer, compass, ambient light sensor, proximity sensor, etc., which allow the combination of inputs and enrichment of HCII interactions [21] As discussed above, the essential functions of HCII are based on a clear signal of emotional state to infer a person's emotional state [30]. Emotions are complex processes comprised of numerous components, including feelings, body changes, cognitive reactions, behavior, and thoughts [41].…”

Section: Backgrounds and Related Work 21 Human-computer Intelligent Interaction (Hcii)mentioning

confidence: 99%

Sensors and Artificial Intelligence Methods and Algorithms for Human–Computer Intelligent Interaction: A Systematic Mapping Study

Šumak

Brdnik

Pušnik

2021

To equip computers with human communication skills and to enable natural interaction between the computer and a human, intelligent solutions are required based on artificial intelligence (AI) methods, algorithms, and sensor technology. This study aimed at identifying and analyzing the state-of-the-art AI methods and algorithms and sensors technology in existing human–computer intelligent interaction (HCII) research to explore trends in HCII research, categorize existing evidence, and identify potential directions for future research. We conduct a systematic mapping study of the HCII body of research. Four hundred fifty-four studies published in various journals and conferences between 2010 and 2021 were identified and analyzed. Studies in the HCII and IUI fields have primarily been focused on intelligent recognition of emotion, gestures, and facial expressions using sensors technology, such as the camera, EEG, Kinect, wearable sensors, eye tracker, gyroscope, and others. Researchers most often apply deep-learning and instance-based AI methods and algorithms. The support sector machine (SVM) is the most widely used algorithm for various kinds of recognition, primarily an emotion, facial expression, and gesture. The convolutional neural network (CNN) is the often-used deep-learning algorithm for emotion recognition, facial recognition, and gesture recognition solutions.