Learning Dynamic Relationships for Facial Expression Recognition Based on Graph Convolutional Network

Jin, Xing; Lai, Zhihui

doi:10.1109/tip.2021.3101820

Cited by 23 publications

(11 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For GNNs including GCNs and GATs, existing methods typically add a limited number of layers due to relatively small graph dimensions and potential overfitting issues. Thus, their computational burden is not significant compared to the overall framework, as has been confirmed by experiments in a few related studies [81,100].…”

Section: Discussionmentioning

confidence: 54%

“…Similarly, [82] also employed landmarkbased ROIs, but the KNN graph was generated in opticalflow space to encode the local manifold structure for a sparse representation [77]. Due to chained reactions among multiple AUs and the symmetrical structure of the human face, [81] proposed a parts-based graph that had manually linked edges by taking FACS and landmarks as references. The nodes were ROIs with Local Binary Pattern (LBP) [20] or deep features as attributes.…”

Section: Region-level Graphsmentioning

confidence: 99%

“…In their relational reasoning, features of each node were generated with its neighbor nodes in the current frame and consecutive frames by using spatial graph convolution and temporal convolution, respectively. Spectral GCN [81]; (e) GAT [89]. Zoom in for better view.…”

Section: Graph Convolutional Networkmentioning

confidence: 99%

“…Alternatively, the approach of spectral GCN [137] has also been studied [113]. [81] devised a lightweight GCN following the Message Passing Neural Network [138]. A learnable adjacency matrix was adapted to infer the spatial dependencies of ROI nodes in different facial affects.…”

Section: Graph Convolutional Networkmentioning

confidence: 99%

“…From the view of the representation, spatial graphs and spatio-temporal graphs are dominant. Specifically, handcrafted features (e.g., LBP [81,83], Gabor [22,115], HOG [36,87,104]) or deep-based features (e.g., CNN [74,115], VGG [91], ResNet [39]) are employed to enhance the node representation similar to many non-graph FER methods [20,31]. For reasoning approaches, early studies prefer to capture the relations of an individual node from predefined graph structures using tracking strategies (e.g., displacement projection [76] and DNG [90]) or general machine learning models (e.g., RF [40], RNN [22], CNN [36]).…”

Section: Facial Expression Recognitionmentioning

confidence: 99%

See 4 more Smart Citations

Graph-Based Facial Affect Analysis: A Review

Liu

Zhang

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This paper provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

show abstract

Section: Discussionmentioning

confidence: 54%