Deep Optimized Priors for 3D Shape Modeling and Reconstruction

Yang, Mingyue; Wen, Yuxin; Chen, Weikai; Chen, Yongwei; Jia, Kui

doi:10.1109/cvpr46437.2021.00328

Cited by 24 publications

(17 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the most recent implicit methods [79], [80] allow the priors of model parameters to be further optimized when fitting to the observed points, i.e., optimizing over both the latent code and model parameters in Eq. (11) or Eq.…”

Section: Learning-based Priorsmentioning

confidence: 99%

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Huang¹,

Wen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reconstruction of a continuous surface of two-dimensional manifold from its raw, discrete point cloud observation is a long-standing problem in computer vision and graphics research. The problem is technically ill-posed, and becomes more difficult considering that various sensing imperfections would appear in the point clouds obtained by practical depth scanning. In literature, a rich set of methods has been proposed, and reviews of existing methods are also provided. However, existing reviews are short of thorough investigations on a common benchmark. The present paper aims to review and benchmark existing methods in the new era of deep learning surface reconstruction. To this end, we contribute a large-scale benchmarking dataset consisting of both synthetic and real-scanned data; the benchmark includes object-and scene-level surfaces and takes into account various sensing imperfections that are commonly encountered in practical depth scanning. We conduct thorough empirical studies by comparing existing methods on the constructed benchmark, and pay special attention on robustness of existing methods against various scanning imperfections; we also study how different methods generalize in terms of reconstructing complex surface shapes. Our studies help identify the best conditions under which different methods work, and suggest some empirical findings. For example, while deep learning methods are increasingly popular in the research community, our systematic studies suggest that, surprisingly, a few classical methods perform even better in terms of both robustness and generalization; our studies also suggest that the practical challenges of misalignment of point sets from multi-view scanning, missing of surface points, and point outliers remain unsolved by all the existing surface reconstruction methods. We expect that the benchmark and our studies would be valuable both for practitioners and as a guidance for new innovations in future research. We make the benchmark publicly accessible at https://Gorilla-Lab-SCUT.github.io/SurfaceReconstructionBenchmark.

show abstract

Section: Learning-based Priorsmentioning

confidence: 99%

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Huang¹,

Wen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This work focuses on soft robot shape representation through a triangle-based mesh to ensure compatibility with modern graphics engines. In the broader domain of shape reconstruction, certain computational techniques have been applied to reconstruct meshes from differing data sources, including 3D pointclouds ( Yang et al, 2021 , or 2D image inputs ( Kolotouros et al, 2019 ; Nguyen et al, 2022 )). Our method relies on skeletal animation to visualize the deformed soft robot shape from the sensor data in real-time.…”

Section: Related Workmentioning

confidence: 99%

Framework for Armature-Based 3D Shape Reconstruction of Sensorized Soft Robots in eXtended Reality

et al. 2022

View full text Add to dashboard Cite

Soft robots are typically intended to operate in highly unpredictable and unstructured environments. Although their soft bodies help them to passively conform to their environment, the execution of specific tasks within such environments often requires the help of an operator that supervises the interaction between the robot and its environment and adjusts the actuation inputs in order to successfully execute the task. However, direct observation of the soft robot is often impeded by the environment in which it operates. Therefore, the operator has to depend on a real-time simulation of the soft robot based on the signals from proprioceptive sensors. However, the complicated three-dimensional (3D) configurations of the soft robot can be difficult to interpret using traditional visualization techniques. In this work, we present an open-source framework for real-time 3D reconstruction of soft robots in eXtended Reality (Augmented and Virtual Reality), based on signals from their proprioceptive sensors. This framework has a Robot Operating System (ROS) backbone, allowing for easy integration with existing soft robot control algorithms for intuitive and real-time teleoperation. This approach is demonstrated in Augmented Reality using a Microsoft Hololens device and runs at up to 60 FPS. We explore the influence that system parameters such as mesh density and armature complexity have on the reconstruction's key performance metrics (i.e., speed, scalability). The open-source framework is expected to function as a platform for future research and developments on real-time remote control of soft robots operating in environments that impede direct observation of the robot.

show abstract

“…Deep Shape Prior. Beside the priors reviewed above, shape priors can also be captured by parameters in neural networks in shape reconstruction [3,16,20,21,25,28,34,65,74,82], segmentation [45,60,61], and completion [32,33,76,80,84]. Deep manifold prior [17] was introduced to reconstruct 3D shapes starting from random initializations.…”

Section: Related Workmentioning

confidence: 99%

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Baorui¹,

Liu²,

Zwicker³

et al. 2022

Preprint

View full text Add to dashboard Cite

Surface reconstruction from point clouds is vital for 3D computer vision. State-of-the-art methods leverage large datasets to first learn local context priors that are represented as neural network-based signed distance functions (SDFs) with some parameters encoding the local contexts. To reconstruct a surface at a specific query location at inference time, these methods then match the local reconstruction target by searching for the best match in the local prior space (by optimizing the parameters encoding the local context) at the given query location. However, this requires the local context prior to generalize to a wide variety of unseen target regions, which is hard to achieve.

show abstract

Deep Optimized Priors for 3D Shape Modeling and Reconstruction

Cited by 24 publications

References 20 publications

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Framework for Armature-Based 3D Shape Reconstruction of Sensorized Soft Robots in eXtended Reality

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Contact Info

Product

Resources

About