LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation

Park, Keunhong; Mousavian, Arsalan; Yu, Xi; Fox, Dieter

doi:10.1109/cvpr42600.2020.01072

Cited by 94 publications

(113 citation statements)

References 27 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…These works can generalize to unseen objects, but still require 3D object models. Park et al [19] relax this assumption by estimating a 3D geometric model by learning a 3D object representation that enforces consistency across multiple views. Then, this estimated 3D object model can be rendered as a depth image of the object in a desired pose.…”

Section: Related Workmentioning

confidence: 99%

Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms

Devgon

Ichnowski

Balakrishna

et al. 2020

2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

Orienting objects is a critical component in the automation of many packing and assembly tasks. We present an algorithm to orient novel objects given a depth image of the object in its current and desired orientation. We formulate a self-supervised objective for this problem and train a deep neural network to estimate the 3D rotation as parameterized by a quaternion, between these current and desired depth images. We then use the trained network in a proportional controller to re-orient objects based on the estimated rotation between the two depth images. Results suggest that in simulation we can rotate unseen objects with unknown geometries by up to 30°with a median angle error of 1.47°over 100 random initial/desired orientations each for 22 novel objects. Experiments on physical objects suggest that the controller can achieve a median angle error of 4.2°over 10 random initial/desired orientations each for 5 objects.

show abstract

Section: Related Workmentioning

confidence: 99%

Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms

Devgon

Ichnowski

Balakrishna

et al. 2020

2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

show abstract

“…Thus, it allows for directly synthesizing the object appearance, eliminating the intermediate step of reconstructing the object in 3D. Latent-Fusion [37] proposes a 3D latent space based object representation for unseen object pose estimation. In contrast to ours, it requires multi-view imagery of the test object to form the latent space and depth measurements at test time.…”

Section: Object Pose Estimationmentioning

confidence: 99%

“…In contrast to ours, it requires multi-view imagery of the test object to form the latent space and depth measurements at test time. In contrast to both NOCS [50] and LatentFusion [37], our model enables 3D object pose estimation from a single RGB image as input.…”

Section: Object Pose Estimationmentioning

confidence: 99%

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

Chen

Dong

Song

et al. 2020

Computer Vision – ECCV 2020

View full text Add to dashboard Cite

Many object pose estimation algorithms rely on the analysisby-synthesis framework which requires explicit representations of individual object instances. In this paper we combine a gradient-based fitting procedure with a parametric neural image synthesis module that is capable of implicitly representing the appearance, shape and pose of entire object categories, thus rendering the need for explicit CAD models per object instance unnecessary. The image synthesis network is designed to efficiently span the pose configuration space so that model capacity can be used to capture the shape and local appearance (i.e., texture) variations jointly. At inference time the synthesized images are compared to the target via an appearance based loss and the error signal is backpropagated through the network to the input parameters. Keeping the network parameters fixed, this allows for iterative optimization of the object pose, shape and appearance in a joint manner and we experimentally show that the method can recover orientation of objects with high accuracy from 2D images alone. When provided with depth measurements, to overcome scale ambiguities, the method can accurately recover the full 6DOF pose successfully.

show abstract

“…Early works [10,11] directly apply 2D CNN on RGB-D image for 6DoF pose regression, while 2D CNN does not characterize 3D geometric information well. To better explore the 3D geometric information, the 3D space corresponding to depth image is divided into voxel grids, and then 3D CNN is applied on voxels [18][19][20]. Higher-dimensional convolution on 3D voxel requires huge computational resources.…”

Section: Related Workmentioning

confidence: 99%

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Chen

Yao

et al. 2020

Sensors

View full text Add to dashboard Cite

6DoF object pose estimation is a foundation for many important applications, such as robotic grasping, automatic driving, and so on. However, it is very challenging to estimate 6DoF pose of transparent object which is commonly seen in our daily life, because the optical characteristics of transparent material lead to significant depth error which results in false estimation. To solve this problem, a two-stage approach is proposed to estimate 6DoF pose of transparent object from a single RGB-D image. In the first stage, the influence of the depth error is eliminated by transparent segmentation, surface normal recovering, and RANSAC plane estimation. In the second stage, an extended point-cloud representation is presented to accurately and efficiently estimate object pose. As far as we know, it is the first deep learning based approach which focuses on 6DoF pose estimation of transparent objects from a single RGB-D image. Experimental results show that the proposed approach can effectively estimate 6DoF pose of transparent object, and it out-performs the state-of-the-art baselines by a large margin.

show abstract

LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation

Cited by 94 publications

References 27 publications

Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms

Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Contact Info

Product

Resources

About