A comparative study on movement feature in different directions for micro-expression recognition

Wei, Jinsheng; Lu, Guanming; Yan, Jun

doi:10.1016/j.neucom.2021.03.063

Cited by 14 publications

(3 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The Local Binary Pattern Six Interception Points (LBP-SIP) ( Wang et al, 2014 ) and Local Binary Pattern from Mean Orthogonal Planes (LBP-MOP) ( Wang et al, 2015 ) are used to reduce the redundancy problem. The Kernelized Two-Groups Sparse Learning (KTGSL) ( Wei et al, 2022b ) automatically learns more discriminative features from Local Binary Pattern with Single Direction Gradient (LBP-SDG) ( Wei et al, 2021 ) and Local Binary Pattern from Five Intersecting Planes (LBP-FIP) ( Wei et al, 2022a ) two sets of features to improve micro-expression recognition performance. The Discriminative Spatiotemporal Local Radon Binary Pattern Based on Revisited Integral Projection (DiSTLBP-RIP) ( Huang et al, 2019 ) fuses shape features into LBP-TOP to improve the ability to discriminate micro-expressions.…”

Section: Related Researchmentioning

confidence: 99%

Multi-scale fusion visual attention network for facial micro-expression recognition

et al. 2023

View full text Add to dashboard Cite

IntroductionMicro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest.MethodsThis paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the micro-expression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for micro-expression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model.ResultsThe proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.DiscussionThis paper proposed MFVAN model is the first to combine image generation with visual attention mechanisms to solve the combination challenge problem of individual identity attribute interference and low-intensity facial muscle movements. Meanwhile, the MFVAN model reveal the impact of individual attributes on the localization of local ROIs. The experimental results show that a multi-scale fusion visual attention network contributes to micro-expression recognition.

show abstract

Section: Related Researchmentioning

confidence: 99%

Multi-scale fusion visual attention network for facial micro-expression recognition

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Одне нещодавнє дослідження Jinsheng Wei та ін. (2021) [9] пропонує новий підхід для розпізнавання мікровиразів обличчя за допомогою ансамблю глибоких згорткових нейронних мереж (CNN). Запропонований підхід використовує етап попередньої обробки для виявлення цікавих областей на обличчі та виділення відповідних рис обличчя.…”

Section: постановка проблемиunclassified

Research of Structural and Mechanical Properties of Meat as an Object of Processing in Meat Comminutor

YAREMCHENKO,

PUKACH

2023

Herald KhmNU.TS

View full text Add to dashboard Cite

This paper proposes the use of micro mimics, small facial muscle movements that are difficult to detect with the naked eye, for the assessment of psychological states using artificial intelligence. The aim of the research is to develop and improve methods for the analysis of micro mimics to accurately recognize emotions and psychological states of individuals. In this study, we conducted an experimental investigation of the proposed method using video recordings of individuals in various emotional states. Additionally, our approach has several advantages over previous methods in micro mimics analysis. For instance, our method does not rely on manual annotation, which is time-consuming and prone to human error. Instead, our approach uses an automated process that is more efficient and consistent. In comparison with the previous studies, the proposed method has several advantages, including real-time analysis of micro mimics, the capability to handle large-scale datasets, and the ability to analyze complex facial expressions. Moreover, the proposed method overcomes the limitations of traditional machine learning algorithms, which are less effective in capturing the temporal dependencies in video data. Overall, the proposed method has demonstrated promising results in micro mimics analysis and provides a solid foundation for future research in the field of emotional AI. In conclusion, the results of our experiment and their comparison with the results of previous research demonstrate the effectiveness and applicability of our proposed method in analyzing micro mimics for assessing the psychological state using deep learning algorithms. This study contributes to the field of emotional AI and opens up new opportunities for the assessment of psychological states using micro mimics. The results of this study could be useful in a variety of applications, including mental health, human-computer interaction, and social robotics.

show abstract

“…The first spontaneous MER research can be traced to Pfister et al's work [14] which utilized a Local Binary Pat- tern from Three Orthogonal Planes (LBP-TOP) [15] on the first public spontaneous ME dataset: SMIC [16]. Following the work of [15], various approaches based on appearance and geometry features [17], [18] were proposed for improving the performance of MER.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

Self Cite

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

A comparative study on movement feature in different directions for micro-expression recognition

Cited by 14 publications

References 40 publications

Multi-scale fusion visual attention network for facial micro-expression recognition

Multi-scale fusion visual attention network for facial micro-expression recognition

Research of Structural and Mechanical Properties of Meat as an Object of Processing in Meat Comminutor

Deep Learning for Micro-Expression Recognition: A Survey

Contact Info

Product

Resources

About