Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

Liu, Zhilei; Dong, Jiahui; Zhang, Cuicui; Wang, Longbiao; Dang, Jianwu

doi:10.1007/978-3-030-37734-2_40

Cited by 61 publications

(40 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Facial analysis: Clinical experts rely on certain facial modifications and symptoms for assistive medical diagnosis, and computer vision has been introduced to offer an automatic and objective assessment of facial features. Interesting results have been obtained by incorporating graph-based models for facial expression recognition [ 234 ], action unit detection [ 235 ] and micro-expression recognition [ 236 ].…”

Section: Research Challenges and Future Directionsmentioning

confidence: 99%

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

Ahmedt-Aristizabal

Armin

Denman

et al. 2021

Sensors

134

View full text Add to dashboard Cite

With the advances of data-driven machine learning research, a wide variety of prediction problems have been tackled. It has become critical to explore how machine learning and specifically deep learning methods can be exploited to analyse healthcare data. A major limitation of existing methods has been the focus on grid-like data; however, the structure of physiological recordings are often irregular and unordered, which makes it difficult to conceptualise them as a matrix. As such, graph neural networks have attracted significant attention by exploiting implicit information that resides in a biological system, with interacting nodes connected by edges whose weights can be determined by either temporal associations or anatomical junctions. In this survey, we thoroughly review the different types of graph architectures and their applications in healthcare. We provide an overview of these methods in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure, and electrical-based analysis. We also outline the limitations of existing techniques and discuss potential directions for future research.

show abstract

Section: Research Challenges and Future Directionsmentioning

confidence: 99%

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

Ahmedt-Aristizabal

Armin

Denman

et al. 2021

Sensors

134

View full text Add to dashboard Cite

show abstract

“…Since the occurrence of AUs are strongly correlated, AU detection is usually considered as a multi-label learning problem. Several works considered the relationships among AUs and modeled AU interrelations to improve recognition accuracy [153], [154], [155]. However, most works rely on probabilistic graphical models with manually extracted features [156], [157].…”

Section: Recognition Based On Facial Action Unitsmentioning

confidence: 99%

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Xie¹,

Lo²,

Shuai³

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In comparison to macro-expressions, micro-expressions are more challenging to be analyzed due to the short span of time and the fine-grained changes. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications, e.g. police interrogation, clinical diagnosis, depression analysis, and business negotiation. In this survey, we offer a fresh overview to discuss new research directions and challenges these days for MER tasks. For example, we review MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units. Moreover, to mitigate the problem of limited and biased ME data, synthetic data generation is surveyed for the diversity enrichment of micro-expression data. Since micro-expression spotting can boost micro-expression analysis, the state-of-the-art spotting works are also introduced in this paper. At last, we discuss the challenges in MER research and provide potential solutions as well as possible directions for further investigation.

show abstract

“…There are many works about AU occurrence detection [8,22,23,29], which is considered as a multilabel classification problem. For instance, several works considered the relationships on various AUs and modeled AU interrelations to improve recognition accuracy [4,21,26]. However, most works relied on probabilistic graphical models with manually extracted features [5,53], which limits the extension for deep learning.…”

Section: Facial Au Detectionmentioning

confidence: 99%

“…However, most works relied on probabilistic graphical models with manually extracted features [5,53], which limits the extension for deep learning. Given that the graph has the natural ability of handling multi-relational data [4], Liu et al [26] proposed the first work that employed GCN to model AU relationship. The cropped AU regions by EAC-Net [23] were fed into GCN as nodes, after that the propagation of the graph was determined by the relationship of AUs.…”

Section: Facial Au Detectionmentioning

confidence: 99%

AU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition

Xie

Shuai

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

115

111

View full text Add to dashboard Cite

Micro-expressions (MEs) are important clues for reflecting the real feelings of humans, and micro-expression recognition (MER) can thus be applied in various real-world applications. However, it is difficult to perceive and interpret MEs correctly. With the advance of deep learning technologies, the accuracy of micro-expression recognition is improved but still limited by the lack of large-scale datasets. In this paper, we propose a novel micro-expression recognition approach by combining Action Units (AUs) and emotion category labels. Specifically, based on facial muscle movements, we model different AUs based on relational information and integrate the AUs recognition task with MER. Besides, to overcome the shortcomings of limited and imbalanced training samples, we propose a data augmentation method that can generate nearly indistinguishable image sequences with AU intensity of real-world micro-expression images, which effectively improve the performance and are compatible with other micro-expression recognition methods. Experimental results on three mainstream micro-expression datasets, i.e., CASME II, SAMM, and SMIC, manifest that our approach outperforms other state-of-the-art methods on both single database and cross-database micro-expression recognition. CCS CONCEPTS • Computing methodologies → Neural networks; Computer vision; Image representations.

show abstract

Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

Cited by 61 publications

References 23 publications

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

AU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition

Contact Info

Product

Resources

About