PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation

He, Yisheng; Sun, Wei; Huang, Haibin; Liu, Jianran; Fan, Haoqiang; Sun, Jian

doi:10.48550/arxiv.1911.04231

Cited by 11 publications

(11 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter can also be verified inFigures 20,21,23,24 and 22 present our collected trajectories in various buildings for the train (20, 21 with red trajectories), test (23, 24 with yellow trajectories) and validation (22 with green trajectories) sets respectively. The gamification approach allowed us to collect realistic trajectories enabling our dataset to be used in different computer vision, and robotic tasks (i.e.…”

supporting

confidence: 61%

“…PVNet [48] votes for keypoints, which are then used to estimate the object's 6DOF pose using PnP. Following a dense correspondence approach, various works directly regress either 2D image to 3D model correspondences [71], normalized object coordinates [65], or leverage deep Hough voting as in the case of PVN3D [23]. Regarding coordinates based regression, separate branches for the translation and rotational components were found to be more robust in predict pose [36], especially for occluded and textureless objects.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DronePose: Photorealistic UAV-Assistant Dataset Synthesis for 3D Pose Estimation via a Smooth Silhouette Loss

Albanis¹,

Zioulis²,

Dimou³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this work we consider UAVs as cooperative agents supporting human users in their operations. In this context, the 3D localisation of the UAV assistant is an important task that can facilitate the exchange of spatial information between the user and the UAV. To address this in a data-driven manner, we design a data synthesis pipeline to create a realistic multimodal dataset that includes both the exocentric user view, and the egocentric UAV view. We then exploit the joint availability of photorealistic and synthesized inputs to train a single-shot monocular pose estimation model. During training we leverage differentiable rendering to supplement a state-of-the-art direct regression objective with a novel smooth silhouette loss. Our results demonstrate its qualitative and quantitative performance gains over traditional silhouette objectives. Our data and code are available at https://vcl3d.github.io/DronePose.

show abstract

supporting

confidence: 61%

Section: Related Workmentioning

confidence: 99%

DronePose: Photorealistic UAV-Assistant Dataset Synthesis for 3D Pose Estimation via a Smooth Silhouette Loss

Albanis¹,

Zioulis²,

Dimou³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…More recent approaches regress object 3D coordinates at each pixel [27,39] in a normalized space to then fit the pose of the objects using their 3D representations. Finally, PVN3D [15] relies on deep Hough voting to regress 3D keypoints directly and then fits the pose through least squares optimization. In this work, we bridge the task of semantic segmentation of an object and that of pose estimation by adding geometric constraints during training of the segmentation network.…”

Section: Related Workmentioning

confidence: 99%

Deep Soft Procrustes for Markerless Volumetric Sensor Alignment

Sterzentsenko¹,

Doumanoglou²,

Thermos³

et al. 2020

Preprint

View full text Add to dashboard Cite

With the advent of consumer grade depth sensors, low-cost volumetric capture systems are easier to deploy. Their wider adoption though depends on their usability and by extension on the practicality of spatially aligning multiple sensors. Most existing alignment approaches employ visual patterns, e.g. checkerboards, or markers and require high user involvement and technical knowledge. More user-friendly and easier-to-use approaches rely on markerless methods that exploit geometric patterns of a physical structure. However, current SoA approaches are bounded by restrictions in the placement and the number of sensors. In this work, we improve markerless data-driven correspondence estimation to achieve more robust and flexible multi-sensor spatial alignment. In particular, we incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. This is accomplished by a soft, differentiable procrustes analysis that regularizes the segmentation and achieves higher extrinsic calibration performance in expanded sensor placement configurations, while being unrestricted by the number of sensors of the volumetric capture system. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure. Code and pretrained models are available at https://vcl3d.github.io/StructureNet/.

show abstract

“…The depth map captured by RGB-D camera preserves important geometry information of the given scene, allowing 2D algorithms to be extend into 3D space. Depth awareness has been proven to be crucial for many applications of scene understanding, e.g., scene parsing [62,30], 6D object pose estimation [59,28] and object detection [25,50], leading to a significant performance enhancement. Recently, there have been a few attempts to take into account the 3D geometric information for salient object detection in the given scene, e.g., by distilling prior knowledge from the depth [52] or incorporating depth information into a SOD framework [87,49,21].…”

Section: Introductionmentioning

confidence: 99%

Cascade Graph Neural Networks for RGB-D Salient Object Detection

Luo

Li²,

Yang³

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper, we study the problem of salient object detection (SOD) for RGB-D images using both color and depth information. A major technical challenge in performing salient object detection from RGB-D images is how to fully leverage the two complementary data sources. Current works either simply distill prior knowledge from the corresponding depth map for handling the RGB-image or blindly fuse color and geometric information to generate the coarse depth-aware representations, hindering the performance of RGB-D saliency detectors. In this work, we introduce Cascade Graph Neural Networks (Cas-Gnn), a unified framework which is capable of comprehensively distilling and reasoning the mutual benefits between these two data sources through a set of cascade graphs, to learn powerful representations for RGB-D salient object detection. Cas-Gnn processes the two data sources individually and employs a novel Cascade Graph Reasoning (CGR) module to learn powerful dense feature embeddings, from which the saliency map can be easily inferred. Contrast to the previous approaches, the explicitly modeling and reasoning of high-level relations between complementary data sources allows us to better overcome challenges such as occlusions and ambiguities. Extensive experiments demonstrate that Cas-Gnn achieves significantly better performance than all existing RGB-D SOD approaches on several widely-used benchmarks. Code is available at https://github.com/LA30/Cas-Gnn.

show abstract

PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation

Cited by 11 publications

References 59 publications

DronePose: Photorealistic UAV-Assistant Dataset Synthesis for 3D Pose Estimation via a Smooth Silhouette Loss

DronePose: Photorealistic UAV-Assistant Dataset Synthesis for 3D Pose Estimation via a Smooth Silhouette Loss

Deep Soft Procrustes for Markerless Volumetric Sensor Alignment

Cascade Graph Neural Networks for RGB-D Salient Object Detection

Contact Info

Product

Resources

About