PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

Papandreou, George; Zhu, Tyler; Chen, Liang-Chieh; Gidaris, Spyros; Tompson, Jonathan; Murphy, Kevin

doi:10.1007/978-3-030-01264-9_17

Cited by 512 publications

(350 citation statements)

References 72 publications

(114 reference statements)

Supporting

Mentioning

328

Contrasting

Order By: Relevance

“…We can see that the proposed SPM model achieves overall 0.669 AP, which is slightly lower than the state-ofthe-art [28]. However, our SPM achieves superior speed, 8× faster than [28]. These results further confirm the superior efficiency of our single-stage solution over existing two-stage top-down or bottom-up strategies, while achieving very competitive performance, for addressing the multiperson pose estimation tasks.…”

Section: Results On Pascal-person-part Datasetsupporting

confidence: 57%

See 1 more Smart Citation

Single-Stage Multi-Person Pose Machines

Nie¹,

Feng²,

Zhang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

221

126

View full text Add to dashboard Cite

Multi-person pose estimation is a challenging problem. Existing methods are mostly two-stage based-one stage for proposal generation and the other for allocating poses to corresponding persons. However, such two-stage methods generally suffer low efficiency. In this work, we present the first single-stage model, Single-stage multi-person Pose Machine (SPM), to simplify the pipeline and lift the efficiency for multi-person pose estimation. To achieve this, we propose a novel Structured Pose Representation (SPR) that unifies person instance and body joint position representations. Based on SPR, we develop the SPM model that can directly predict structured poses for multiple persons in a single stage, and thus offer a more compact pipeline and attractive efficiency advantage over two-stage methods. In particular, SPR introduces the root joints to indicate different person instances and human body joint positions are encoded into their displacements w.r.t. the roots. To better predict long-range displacements for some joints, SPR is further extended to hierarchical representations. Based on SPR, SPM can efficiently perform multi-person poses estimation by simultaneously predicting root joints (location of instances) and body joint displacements via CNNs. Moreover, to demonstrate the generality of SPM, we also apply it to multi-person 3D pose estimation. Comprehensive experiments on benchmarks MPII, extended PASCAL-Person-Part, MSCOCO and CMU Panoptic clearly demonstrate the state-of-the-art efficiency of SPM for multi-person 2D/3D pose estimation, together with outstanding accuracy.

show abstract

Section: Results On Pascal-person-part Datasetsupporting

confidence: 57%

“…Qualitative results are shown in the middle row of Table 4 shows experimental results on MSCOCO testdev. We can see that the proposed SPM model achieves overall 0.669 AP, which is slightly lower than the state-ofthe-art [28]. However, our SPM achieves superior speed, 8× faster than [28].…”

Section: Results On Pascal-person-part Datasetmentioning

confidence: 81%

Single-Stage Multi-Person Pose Machines

Nie¹,

Feng²,

Zhang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

221

126

View full text Add to dashboard Cite

show abstract

“…We use binary cross entropy loss to optimize the parameters. After obtaining the heatmap of the keypoints, we use a single offset map s(x) [8] to extract the local maxima for each heatmap disc on h(x). This can be viewed as a non-maximum suppression (NMS) operation.…”

Section: Bounding Box Generationmentioning

confidence: 99%

Multi-scale Cell Instance Segmentation with Keypoint Graph Based Bounding Boxes

Huang

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Most existing methods handle cell instance segmentation problems directly without relying on additional detection boxes. These methods generally fails to separate touching cells due to the lack of global understanding of the objects. In contrast, box-based instance segmentation solves this problem by combining object detection with segmentation. However, existing methods typically utilize anchor boxbased detectors, which would lead to inferior instance segmentation performance due to the class imbalance issue. In this paper, we propose a new box-based cell instance segmentation method. In particular, we first detect the five pre-defined points of a cell via keypoints detection. Then we group these points according to a keypoint graph and subsequently extract the bounding box for each cell. Finally, cell segmentation is performed on feature maps within the bounding boxes. We validate our method on two cell datasets with distinct object shapes, and empirically demonstrate the superiority of our method compared to other instance segmentation techniques. Code is available at: https: //github.com/yijingru/KG_Instance_Segmentation.

show abstract

“…Several multi-task approaches exist for visual person analysis using a single dataset. Some train pose estimation and part segmentation jointly [23,16]. Hyperface [27] performs face detection, landmark localization, gender classification, and headpose estimation in a single network, but does not consider ReID.…”

Section: Related Workmentioning

confidence: 99%

Visual Person Understanding Through Multi-task and Multi-dataset Learning

Pfeiffer

Hermans

Sárándi

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We address the problem of learning a single model for person re-identification, attribute classification, body part segmentation, and pose estimation. With predictions for these tasks we gain a more holistic understanding of persons, which is valuable for many applications. This is a classical multi-task learning problem. However, no dataset exists that these tasks could be jointly learned from. Hence several datasets need to be combined during training, which in other contexts has often led to reduced performance in the past. We extensively evaluate how the different task and datasets influence each other and how different degrees of parameter sharing between the tasks affect performance. Our final model matches or outperforms its single-task counterparts without creating significant computational overhead, rendering it highly interesting for resource-constrained scenarios such as mobile robotics.

show abstract

PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

Cited by 512 publications

References 72 publications

Single-Stage Multi-Person Pose Machines

Single-Stage Multi-Person Pose Machines

Multi-scale Cell Instance Segmentation with Keypoint Graph Based Bounding Boxes

Visual Person Understanding Through Multi-task and Multi-dataset Learning

Contact Info

Product

Resources

About