DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction

Zhou, Shihao; Jiang, Mengxi; Cai, Shanshan; Lei, Yunqi

doi:10.1145/3474085.3475242

Cited by 9 publications

(6 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As Figure 2 shows, inspired by Refs. [LWL21, ZJCL21] using vertex mask to enhance the robustness of the model in human shape re‐construction, we model the feature fusion as a feature completion task. Specifically, we render the SMPL model conditioned in fixed camera parameters and image resolution.…”

Section: Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

OaIF: Occlusion‐Aware Implicit Function for Clothed Human Re‐construction

Tan

Guan

Zhou

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Clothed human re‐construction from a monocular image is challenging due to occlusion, depth‐ambiguity and variations of body poses. Recently, shape representation based on an implicit function, compared to explicit representation such as mesh and voxel, is more capable with complex topology of clothed human. This is mainly achieved by using pixel‐aligned features, facilitating implicit function to capture local details. But such methods utilize an identical feature map for all sampled points to get local features, making their models occlusion‐agnostic in the encoding stage. The decoder, as implicit function, only maps features and does not take occlusion into account explicitly. Thus, these methods fail to generalize well in poses with severe self‐occlusion. To address this, we present OaIF to encode local features conditioned in visibility of SMPL vertices. OaIF projects SMPL vertices onto image plane to obtain image features masked by visibility. Vertices features integrated with geometry information of mesh are then feed into a GAT network to encode jointly. We query hybrid features and occlusion factors for points through cross attention and learn occupancy fields for clothed human. The experiments demonstrate that OaIF achieves more robust and accurate re‐construction than the state of the art on both public datasets and wild images.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

“…[LWL21] used transformer to capture the long‐distance relationship. DC‐GNet [ZJCL21] models both the positive and negative dependencies between vertices and introduces mask to tackle a shape completion task. But all the methods are proposed to estimate human pose and shape aligned with the observed image without cloth information.…”

Section: Related Workmentioning

confidence: 99%

OaIF: Occlusion‐Aware Implicit Function for Clothed Human Re‐construction

Tan

Guan

Zhou

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Zhao et al [11] proposed the SemGCN model, which combines the channel weights of the implicit prior edges in the learning graph with kernel attributes and significantly improves the convolution ability of the graph. Zhou et al [12] used a deep mesh relation to generate a GCN. An adaptive adjacency matrix was applied to extract the positive and negative relationships between the joints, and significant results were obtained.…”

Section: Gcns and Their Use In Different Domainsmentioning

confidence: 99%

BCCLR: A Skeleton-Based Action Recognition with Graph Convolutional Network Combining Behavior Dependence and Context Clues

Wang,

Xia,

Liu

2024

CMC

View full text Add to dashboard Cite

In recent years, skeleton-based action recognition has made great achievements in Computer Vision. A graph convolutional network (GCN) is effective for action recognition, modelling the human skeleton as a spatiotemporal graph. Most GCNs define the graph topology by physical relations of the human joints. However, this predefined graph ignores the spatial relationship between non-adjacent joint pairs in special actions and the behavior dependence between joint pairs, resulting in a low recognition rate for specific actions with implicit correlation between joint pairs. In addition, existing methods ignore the trend correlation between adjacent frames within an action and context clues, leading to erroneous action recognition with similar poses. Therefore, this study proposes a learnable GCN based on behavior dependence, which considers implicit joint correlation by constructing a dynamic learnable graph with extraction of specific behavior dependence of joint pairs. By using the weight relationship between the joint pairs, an adaptive model is constructed. It also designs a self-attention module to obtain their inter-frame topological relationship for exploring the context of actions. Combining the shared topology and the multi-head self-attention map, the module obtains the context-based clue topology to update the dynamic graph convolution, achieving accurate recognition of different actions with similar poses. Detailed experiments on public datasets demonstrate that the proposed method achieves better results and realizes higher quality representation of actions under various evaluation protocols compared to state-of-the-art methods.

show abstract

“…A bilayer graph is designed in [316] to represent a fine shape and human pose simultaneously, and a GCN is deployed as the encoder to jointly perform pose estimation and mesh regression. DC-GNet [317] takes into account occlusion cases where human bodies are incomplete in images. For human body reconstruction, Pose2Mesh [318] builds on a GCN to estimate vertex positions on the human mesh from a 2D human pose.…”

Section: D Reconstructionmentioning

confidence: 99%

A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective

Chen¹,

Wu²,

Dai³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have gained momentum in graph representation learning and boosted the state of the art in a variety of areas, such as data mining (e.g., social network analysis and recommender systems), computer vision (e.g., object detection and point cloud learning), and natural language processing (e.g., relation extraction and sequence learning), to name a few. With the emergence of Transformers in natural language processing and computer vision, graph Transformers embed a graph structure into the Transformer architecture to overcome the limitations of local neighborhood aggregation while avoiding strict structural inductive biases. In this paper, we present a comprehensive review of GNNs and graph Transformers in computer vision from a task-oriented perspective. Specifically, we divide their applications in computer vision into five categories according to the modality of input data, i.e., 2D natural images, videos, 3D data, vision + language, and medical images. In each category, we further divide the applications according to a set of vision tasks. Such a task-oriented taxonomy allows us to examine how each task is tackled by different GNN-based approaches and how well these approaches perform. Based on the necessary preliminaries, we provide the definitions and challenges of the tasks, in-depth coverage of the representative approaches, as well as discussions regarding insights, limitations, and future directions.

show abstract

DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction

Cited by 9 publications

References 57 publications

OaIF: Occlusion‐Aware Implicit Function for Clothed Human Re‐construction

OaIF: Occlusion‐Aware Implicit Function for Clothed Human Re‐construction

BCCLR: A Skeleton-Based Action Recognition with Graph Convolutional Network Combining Behavior Dependence and Context Clues

A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective

Contact Info

Product

Resources

About