User action and facial expression recognition for error detection system in an ambient assisted environment

Yaddaden, Yacine; Adda, Mehdi; Bouzouane, Abdenour; Gaboury, Sébastien; Bouchard, Bruno

doi:10.1016/j.eswa.2018.06.033

Cited by 45 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a noninvasive and less costly instrument, FERS with AI-based algorithms also reduced the concerns of inter-and intrarater variation in the interview-based manually scored NPI [10,13]. The accuracy of the customized, video-based FERS was 86% in the Karolinska Directed Emotional Faces (KDEF) dataset; the results were comparable to those developed by histograms of oriented gradients (HOG) of facial feature extraction and other convolutional neural network (CNN) models (Table 6) [10,[21][22][23][24][25][26].…”

Section: Discussionmentioning

confidence: 79%

Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system

Chen

Tsai

et al. 2022

Aging

View full text Add to dashboard Cite

Background: Behavioral and psychological symptoms of dementia (BPSD) affect 90% of persons with dementia (PwD), resulting in various adverse outcomes and aggravating care burdens among their caretakers. This study aimed to explore the potential of artificial intelligence-based facial expression recognition systems (FERS) in predicting BPSDs among PwD. Methods: A hybrid of human labeling and a preconstructed deep learning model was used to differentiate basic facial expressions of individuals to predict the results of Neuropsychiatric Inventory (NPI) assessments by stepwise linear regression (LR), random forest (RF) with importance ranking, and ensemble method (EM) of equal importance, while the accuracy was determined by mean absolute error (MAE) and root-mean-square error (RMSE) methods. Results: Twenty-three PwD from an adult day care center were enrolled with ≥ 11,500 FERS data series and 38 comparative NPI scores. The overall accuracy was 86% on facial expression recognition. Negative facial expressions and variance in emotional switches were important features of BPSDs. A strong positive correlation was identified in each model (EM: r = 0.834, LR: r = 0.821, RF: r = 0.798 by the patientwise method; EM: r = 0.891, LR: r = 0.870, RF: r = 0.886 by the MinimPy method), and EM exhibited the lowest MAE and RMSE. Conclusions: FERS successfully predicted the BPSD of PwD by negative emotions and the variance in emotional switches. This finding enables early detection and management of BPSDs, thus improving the quality of dementia care.

show abstract

Section: Discussionmentioning

confidence: 79%

Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system

Chen

Tsai

et al. 2022

Aging

View full text Add to dashboard Cite

show abstract

“…Lastly, the best fold accuracy is used to show the confusion matrix of the final results. The basic ELM parameters setup to test the performance for each feature extractions mentioned before is as: number of hidden neurons={2, 4,8,12,16,20,24,32, 64, 120}, activation function={Sigmoid, Cosine, Hyperbolic Tangent, Linear}. Figure 5(a) shows that basic ELM method is tested using FL, LBP and GLCM feature extractions with its best parameters which tested before.…”

Section: Experiments On Basic Elm Methodsmentioning

confidence: 99%

“…Facial expressions recognition is an important part of studying how humans react to the environment in the field of affective computing which aims to diagnose and measure someone's emotional expression explicitly and connect implicitly [3]. Facial expression recognition has been widely applied in several fields, including facial expression recognition for the detection of system errors in smart environments [4], enhancing the gaming experience [5], intelligent tutoring system [6], cooking experience [7], and so on. Those examples show that facial expressions play an important role in decision making, help the learning process, and also provide an assessment based on an action.…”

Section: Introductionmentioning

confidence: 99%

Feature extraction comparison for facial expression recognition using adaptive extreme learning machine

Wafi

Bachtiar

Utaminingrum

2023

IJECE

View full text Add to dashboard Cite

Facial expression recognition is an important part in the field of affective computing. Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypes emotional expressions such as anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. This paper aims to compare feature extraction methods that are used to detect human facial expression. The study compares the gray level co-occurrence matrix, local binary pattern, and facial landmark (FL) with two types of facial expression datasets, namely Japanese female facial expression (JFFE), and extended Cohn-Kanade (CK+). In addition, we also propose an enhancement of extreme learning machine (ELM) method that can adaptively select best number of hidden neurons adaptive ELM (aELM) to reach its maximum performance. The result from this paper is our proposed method can slightly improve the performance of basic ELM method using some feature extractions mentioned before. Our proposed method can obtain maximum mean accuracy score of 88.07% on CK+ dataset, and 83.12% on JFFE dataset with FL feature extraction.

show abstract

“…Although these approaches are effective methods in extracting spatial information, they fail to capture morphological and contextual variations in the expression process. Recent methods aim to solve this problem by using massive datasets to obtain more efficient features of FER [9][10][11][12][13][14][15]. Some researchers use multimodal fusion to recognize emotions, such as voices, expressions, and actions [16].…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Zhang

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

show abstract

User action and facial expression recognition for error detection system in an ambient assisted environment

Cited by 45 publications

References 29 publications

Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system

Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system

Feature extraction comparison for facial expression recognition using adaptive extreme learning machine

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Contact Info

Product

Resources

About