Facial micro-expression recognition based on the fusion of deep learning and enhanced optical flow

Li, Qiuyu; Shu, Zhan; Xu, Lu; Wu, Congzhong

doi:10.1007/s11042-018-6857-9

Cited by 27 publications

(21 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 lists the performance of benchmark motion-based methods, [6][7][8][9]20,31,32 with the optimized Bi-WOOF. The best reported accuracy is 74.06% over the CASMEII dataset.…”

Section: Discussionmentioning

confidence: 99%

“…In, 6 authors proposed an optical flow features from Apex-frame network (OFF-ApexNet), which combines optical flow guided context with the convolutional neural network (CNN) to compute features. Then, authors in 7 presented a novel algorithm that combines a deep multi-task convolutional network for detecting facial landmarks and a fused deep convolutional network for micro-expression features. In another study, 8 authors suggested the Riesz pyramid and a multi-scale steerable Hilbert transform.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

et al. 2021

View full text Add to dashboard Cite

The trend of real-time micro-expression recognition systems has increased with recent advancements in human-computer interaction (HCI) in security and healthcare. Several studies in this field contributed towards recognition accuracy, while few studies look into addressing the computation costs. In this paper, two approaches for micro-expression feature extraction are analyzed for real-time automatic micro-expression recognition. Firstly, motion-based approach, which calculates motion of subtle changes from an image sequence and present as features. Then, secondly, a low computational geometric-based feature extraction technique, a very popular method for facial expression recognition in real-time. These approaches were integrated in a developed system together with a facial landmark detection algorithm and a classifier for real-time analysis. Moreover, the recognition performance were evaluated using SMIC, CASME, CAS(ME)2 and SAMM datasets. The results suggest that the optimized Bi-WOOF (leveraging on motion-based features) yields the highest accuracy of 68.5%, while the full-face graph (leveraging on geometric-based features) yields 75.53% on the SAMM dataset. On the other hand, the optimized Bi-WOOF processes sample at 0.36 seconds and full-face graph processes sample at 0.10 seconds with a 640x480 image size. All experiments were performed on an Intel i5-3470 machine.

show abstract

“…Table 3 lists the performance of benchmark motion-based methods, [6][7][8][9]20,31,32 with the optimized Bi-WOOF. The best reported accuracy is 74.06% over the CASMEII dataset.…”

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A face can be divided into regions of interest using the coordinates of the facial landmarks. The division of the face into 12 regions of interest is suggested in [18]. Regions are analyzed for changes in the intensity of each pixel using histograms.…”

Section: Facial Featuresmentioning

confidence: 99%

Facial Expression Recognition using Distance Importance Scores Between Facial Landmarks

Ryumina

Karpov

2020

Proceedings of the 30th International Conference on Computer Graphics and Machine Vision (GraphiCon 2020). Part 2

View full text Add to dashboard Cite

In this paper, we present a feature extraction approach for facial expressions recognition based on distance importance scores between the coordinates of facial landmarks. Two audio-visual speech databases (CREMA-D and RAVDESS) were used in the research. We conducted experiments using the Long Short-Term Memory Recurrent Neural Network model in a single corpus and cross-corpus setup with different length sequences. Experiments were carried out using different sets and types of visual features. An accuracy of facial expression recognition was 79.1% and 98.9% for the CREMA-D and RAVDESS databases, respectively. The extracted features provide a better recognition result compared to other methods based on the analysis of facial graphical regions.

show abstract

“…LK algorithm uses least squares estimation method to solve the overconstrained problem of formula ( 9) by minimizing ‖A d − b‖ 2 . e standard form of LK algorithm is as follows:…”

Section: Lucas-kanade (Lk) Optical Flow Algorithmsmentioning

confidence: 99%

“…However, most of the research on expression focuses on traditional expressions, i.e., macroexpressions or full-expressions, while less on microexpressions. Until recently, more and more scholars began to pay attention to the study of microexpressions [2]. Literature [3] focuses on such a meaningful topic and investigates how to make full advantage of the color information provided by the microexpression samples to deal with the microexpression recognition (MER) problem.…”

Section: Introductionmentioning

confidence: 99%

Enhanced Forest Microexpression Recognition Based on Optical Flow Direction Histogram and Deep Multiview Network

Wang

2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In order to recognize the instantaneous changes of facial microexpressions in natural environment, a method based on optical flow direction histogram and depth multiview network to enhance forest microexpression recognition was proposed. In the preprocessing stage, the histogram equalization of the acquired face image is performed, and then the dense key points of the face are detected. According to the coordinates of the key points and the face action coding system (FACS), the face region is divided into 15 regions of interest (ROI). In the feature extraction stage, the optical flow direction histogram feature between adjacent frames in ROI is extracted to detect the peak frame of microexpression sequence. Finally, the average optical flow direction histogram feature of the image sequence from the initial frame to the peak frame is extracted. In the classification stage, firstly, the head pose parameters under horizontal degrees of freedom are estimated to eliminate the influence of head pose motion, and a forest multiview conditional probability model based on deep multiview network is established. Conditional probability and neural connection function are introduced into the node splitting learning of random tree to improve the learning ability and distinguishing ability of the model on the limited training set. Finally, multiview-weighted voting is used to determine the categories of facial microexpressions. Experiments on CASME II microexpression dataset show that the proposed method can effectively describe the changes of microexpressions and improve the recognition accuracy compared with other new methods.

show abstract

Facial micro-expression recognition based on the fusion of deep learning and enhanced optical flow

Cited by 27 publications

References 44 publications

Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

Facial Expression Recognition using Distance Importance Scores Between Facial Landmarks

Enhanced Forest Microexpression Recognition Based on Optical Flow Direction Histogram and Deep Multiview Network

Contact Info

Product

Resources

About