Transferring pose and augmenting background for deep human-image parsing and its applications

Abstract

Parsing of human images is a fundamental task for determining semantic parts such as the face, arms, and legs, as well as a hat or a dress. Recent deep-learning-based methods have achieved significant improvements, but collecting training datasets with pixel-wise annotations is labor-intensive. In this paper, we propose two solutions to cope with limited datasets. Firstly, to handle various poses, we incorporate a pose estimation network into an end-to-end humanimage parsing network, in order to transfer common features across the domains. The pose estimation network can be trained using rich datasets and can feed valuable features to the human-image parsing network. Secondly, to handle complicated backgrounds, we increase the variation in image backgrounds automatically by replacing the original backgrounds of human images with others obtained from large-scale scenery image datasets. Individually, each solution is versatile and beneficial to human-image parsing, while their combination yields further improvement. We demonstrate the effectiveness of our approach through comparisons and various applications such as garment recoloring, garment texture transfer, and visualization for fashion analysis.

JMNet: A joint matting network for automatic human matting

“…DCNN-based pose estimation methods either use regression to determine the positions of keypoints [20,21], or predict heatmaps generated by the keypoints [22][23][24]. Some papers use pose heatmaps to guide human parsing [25,26] and human image completion [27]. Our method uses heatmaps as a pose representation, because heatmaps can readily encode body structures and provide convenient inputs to the other neural networks.…”

Accurate Human Parsing Model by Edge Contour and Pose Feature

“…基于多任务学习的方 法能够有效地结合多个子任务, 以相互辅助的方 式提高解析的精度. Li 等 [3] 提出结合人体解析和边 缘检测的多任务网络, 通过结合边缘特征和人体 部位特征提高解析结果的精度, 实现人体的精确 解析; Kikuchi 等 [6] 将姿态估计网络合并到端到端 的人体解析网络中, 以便跨域传递姿态的特征, 实 现了姿态特征辅助人体解析, 以获得精确的解析 结果.…”