A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation From a Single Depth Image

Abstract

For 3D hand and body pose estimation task in depth image, a novel anchor-based approach termed Anchor-to-Joint regression network (A2J) with the end-to-end learning ability is proposed. Within A2J, anchor points able to capture global-local spatial context information are densely set on depth image as local regressors for the joints. They contribute to predict the positions of the joints in ensemble way to enhance generalization ability. The proposed 3D articulated pose estimation paradigm is different from the state-of-the-art encoder-decoder based FCN, 3D CNN and point-set based manners. To discover informative anchor points towards certain joint, anchor proposal procedure is also proposed for A2J. Meanwhile 2D CNN (i.e., ResNet-50) is used as backbone network to drive A2J, without using time-consuming 3D convolutional or deconvolutional layers. The experiments on 3 hand datasets and 2 body datasets verify A2J's superiority. Meanwhile, A2J is of high running speed around 100 FPS on single NVIDIA 1080Ti GPU.

AWR: Adaptive Weighting Regression for 3D Hand Pose Estimation

“…We first compare our method with others on HANDS 2017 dataset . Since the HANDS 2017 dataset does not provide test set labels publicly, we evaluate using only mean joint error metric and compare our method with Vanora , THU VCLab (Chen et al 2018a), oasis (Moon, Chang, and Lee 2018a), RCN-3D (Yuan et al 2018), V2V-PoseNet (Moon, Chang, and Lee 2018b) and A2J (Xiong et al 2019). Results in Table 5 reflect that our ResNet18 based method already exceeds previous state-of-the-art methods by a large margin.…”

“…Recent work (Xiong et al 2019) uses 2D offsets between anchor points and hand joints to represent 2D positions of joints. Due to the large variance of offsets, we further decompose them into 2D directional unit vector fields and closeness heatmaps, reflecting 2D directions and closeness from each pixel in depth images to target joints.…”

“…Recently, deep neural network based methods have shown great performance in 3D hand pose estimation (Tompson et al 2014;Ge, Ren, and Yuan 2018;Wang et al 2018;Chen et al 2018a;Moon, Chang, and Lee 2018b;Wan et al 2018;Xiong et al 2019; Zhang and Zhang 2019; . These works can be divided into two categories according to networks' output type regressionbased methods and detection-based methods.…”

“…The idea of aggregating different regions in dense representations to derive joint coordinates has previously been seen in human (Sun et al 2018) or hand pose estimation (Xiong et al 2019; Zhang and Zhang 2019). However, (Sun et al 2018) for human pose estimation focus on RGB images and this type of representation ignores the 3D geometric properties of essentially 2.5D depth images.…”

“…However, (Sun et al 2018) for human pose estimation focus on RGB images and this type of representation ignores the 3D geometric properties of essentially 2.5D depth images. And (Xiong et al 2019; Zhang and Zhang 2019) for hand pose estimation are specialized for certain type of network structure, dense representation and input modality. The effectiveness and generality of this operation are not extensively validated.…”