Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study

Allaert, Benjamin; Ward, Isaac Ronald; Bilasco, Ioan Marius; Djeraba, Chabane; Bennamoun, Mohammed

doi:10.48550/arxiv.1904.11592

Cited by 4 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4) Optical flow: Optical flow is an approximation of the local image motion which has been verified helpful for motion representation [153]. The optical-flow based facial movement analysis [154], [155] offers a promising way for expression analysis with subtle movements [156], [157], various head poses [158] as well as facial occlusions [159]. Optical flow specifies the magnitude and direction of image pixel moves in a given sequence of images with a two-dimension vector field (horizontal and vertical optical flows).…”

Section: B Dynamic Image Sequencementioning

confidence: 99%

Deep Learning for Micro-expression Recognition: A Survey

Li¹,

Wei²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in medical treatment, national security, interrogations and many human-computer interaction systems. Early methods for MER mainly based on traditional appearance and geometry features. Recently, with the success of deep learning (DL) in various fields, neural networks have received increasing interests in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection, thus have small-scale datasets. DL based MER becomes challenging due to above ME characters. To data, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep microexpression recognition (MER), including datasets, deep MER pipeline, and the bench-marking of most influential methods. This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL. For each aspect, the basic approaches and advanced developments are summarized and discussed. In addition, we conclude the remaining challenges and and potential directions for the design of robust deep MER systems. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

Section: B Dynamic Image Sequencementioning

confidence: 99%

Deep Learning for Micro-expression Recognition: A Survey

Li¹,

Wei²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Third, we compute the optical flow from the cropped faces using the Farnebäck [6] method. We have selected the Farnebäck method to compute optical flows as Allaert et al [1] show that this method is particularly adapted for facial expressions. In order to generate normalized optical flows for all images and to reduce the computational cost of the auto-encoder, we resize the optical flows to a smaller size in the x and y dimensions.…”

Section: Optical Flow Calculationmentioning

confidence: 99%

“…Among the different architectures that are proposed in the literature, CNN architectures seem to be the mostly used architectures for static facial expression recognition [10]. In order to add time, Allaert et al [1] proposed to feed a CNN architecture with optical flows. We propose in this paper to build the same architecture as in [1] as it has been already developed in the context of optical flow classification for facial expression recognition.…”

Section: Facial Expression Classificationmentioning

confidence: 99%

“…In order to add time, Allaert et al [1] proposed to feed a CNN architecture with optical flows. We propose in this paper to build the same architecture as in [1] as it has been already developed in the context of optical flow classification for facial expression recognition. This architecture is not necessarily the state-of-the-art architecture but, in this paper, we do not focus on the performance of the recognition classifier but on the ability of our method to compensate under the effect of occlusions on the performance of the classifier.…”

Section: Facial Expression Classificationmentioning

confidence: 99%

“…In spite of these results, most of the solutions to handle occlusions are based on static images. Recent papers tend, in general, to neglect research on motion information as seen in the recent CVPR 2019 statistics 1 . Nevertheless, motion remains an important and really useful information, as demonstrated for example, by the ECCV 2020 best paper award 2 which proposed a new deep learning architecture to calculate optical flow [18].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction

Poux,

Allaert,

Ihaddadene

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Video facial expression recognition is useful for many applications and received much interest lately. Although some solutions give really good results in a controlled environment (no occlusion), recognition in the presence of partial facial occlusion remains a challenging task. To handle occlusions, solutions based on the reconstruction of the occluded part of the face have been proposed. These solutions are mainly based on the texture or the geometry of the face. However, the similarity of the face movement between different persons doing the same expression seems to be a real asset for the reconstruction. In this paper we exploit this asset and propose a new solution based on an auto-encoder with skip connections to reconstruct the occluded part of the face in the optical flow domain. To the best of our knowledge, this is the first proposition to directly reconstruct the movement for facial expression recognition. We validated our approach in the controlled dataset CK+ on which different occlusions were generated. Our experiments show that the proposed method reduce significantly the gap, in terms of recognition accuracy, between occluded and non-occluded situations. We also compare our approach with existing state-of-the-art solutions. In order to lay the basis of a reproducible and fair comparison in the future, we also propose a new experimental protocol that includes occlusion generation and reconstruction evaluation.

show abstract

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study

Cited by 4 publications

References 23 publications

Deep Learning for Micro-expression Recognition: A Survey

Deep Learning for Micro-expression Recognition: A Survey

Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction

Deep Learning for Micro-Expression Recognition: A Survey

Contact Info

Product

Resources

About