Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations

Kim, Dae Hoe; Baddar, Wissam J.; Ro, Yong Man

doi:10.1145/2964284.2967247

Cited by 154 publications

(106 citation statements)

References 12 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…Due to larger number samples in CASME II dataset as compared to CASME dataset, the researchers in [12] opted to demonstrate deep learning results on video clips from CASME II dataset. In our research, we have applied data augmentation technique to increase the number of samples.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Accuracy LBP-TOP+SVM [26] 75.30% MDMO+SVM [19] 67.37% CNN+LSTM [12] 60.98% Proposed CNN method 75.57%…”

Section: Methodsmentioning

confidence: 99%

“…Compare to traditional hand-engineered features, such as Local Binary Pattern (LBP) [31] and Histogram of Gradients (HoG) [1,16]; a deep convolutional neural network consists of multiple layers which can automatically learn hierarchies visual features directly from the raw image pixels. In the research [12], Kim et al introduced deep learning features for micro-expression recognition. They proposed a new method consisting of two sections.…”

Section: Related Workmentioning

confidence: 99%

“…Next, the learned spatial features with expression-state constraints are transferred to learn temporal features of microexpression. The temporal feature learning converts the temporal characteristics of the different states of the microexpression using long short-term memory (LSTM) recurrent neural networks (RNNs) [12]. The time scale dependent information that resides along the video sequences is consequently learned by using LSTM.…”

Section: Related Workmentioning

confidence: 99%

“…To address this issue, a large number of strategies have been proposed: fine-tuning models trained from other large public datasets (e.g. ImageNet [2]), using the big synthetic training datasets explored by some authors [8,12,14].…”

Section: A Data Augmentationmentioning

confidence: 99%

See 4 more Smart Citations

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Takalkar

2017

2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

View full text Add to dashboard Cite

Abstract-Facial micro-expression refers to split-second muscle changes in the face, indicating that a person is either consciously or unconsciously suppressing their true emotions and even mental health. Therefore, micro-expression recognition attracts increasing research efforts in both fields of psychology and computer vision. Existing research on micro-expression recognition has mainly used hand-crafted features, for example, Local Binary Pattern-Three Orthogonal Planes (LBP-TOP), Gabor filter and optical flow. Recently, Deep Convolutional neural systems have demonstrated a high degree effectiveness for difficult face recognition tasks. This paper explores the possible use of deep learning for micro-expression recognition. To develop a reliable deep neural network extensive training sets are required with a huge number of labeled image samples. However, micro-expression recognition is a challenging task due to the repressed facial appearance and short duration, which results in the lack of training data. In this paper, we propose to generate extensive training datasets of synthetic images using data augmentation on CASME and CASME II databases. Then, these datasets are combined to tune a satisfactory CNN-based microexpression recognizer. Experimental results demonstrate the effectiveness of the proposed CNN approach in image based micro-expression recognition and present comparable results with the best-related works.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

“…Accuracy LBP-TOP+SVM [26] 75.30% MDMO+SVM [19] 67.37% CNN+LSTM [12] 60.98% Proposed CNN method 75.57%…”

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A Data Augmentationmentioning

confidence: 99%

See 3 more Smart Citations

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Takalkar

2017

2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

View full text Add to dashboard Cite

show abstract

Industrial product design method using feature expression and multi‐view learning in ICN

Fang

2022

Internet Technology Letters

View full text Add to dashboard Cite

In order to realize multi‐layer semantic integration and innovation of complex product industrial design, promote multi‐disciplinary knowledge sharing and coordination, and improve product quality, the hierarchical semantic concept of product industrial design is put forward based on the theories of product semiotics and cognitive semantics. In addition, the hierarchical semantic industrial design process context model is constructed based on complex product development process and information processing theory using the Information Centric Networking (ICN) technology, since ICN architecture can be used to optimize the product quality. In the ICN environment, the scientific semantic expression methods and the artistic semantic expression method of product semantics are also used to extract features and construct the feature expression in the specific design situation. The icon semantic features, indicative semantic features, symbolic semantic features and their relevance of complex products are obtained. Experimental results have been achieved and shown that the proposed method has benefits than the other methods.

show abstract

TSNN:Three‐StreamCombining2Dand3DConvolutional Neural Network for Micro‐Expression Recognition

Guo

2020

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Non-member Fan Guo a , Non-member Facial micro-expression recognition is a natural mechanism of facial behavior with subtle muscle movements and short duration, which is widely considered to be hard to recognize. In this paper, we propose the temporal sampling deformation (TSD) to normalize the temporal lengths and conserve time domain information for micro-expression sequences. A three-stream combining 2D and 3D convolutional neural network (TSNN) is also proposed to capture the features of micro-expressions and classify the expressions as well. The proposed network has two variants TSNN-IF and TSNN-LF, which can automatically learn spatial and temporal features at the same time. Single domain experiments and cross-domain experiments are also performed in the three benchmark datasets (chinese academy of sciences micro-expression II (CASME II), spontaneous micro-expression database (SMIC), and spontaneous micro-facial movement dataset (SAMM)) to verify the effectiveness and validity of the proposed framework. Comprehensive results and ablation studies show that the proposed method can achieve comparable or even better results compared with other state-of-the-art methods for micro-expression recognition.

show abstract

Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations

Cited by 154 publications

References 12 publications

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Industrial product design method using feature expression and multi‐view learning in ICN

TSNN:Three‐StreamCombining2Dand3DConvolutional Neural Network for Micro‐Expression Recognition

Contact Info

Product

Resources

About