CuFusion: Accurate Real‐time Camera Tracking and Volumetric Scene Reconstruction with a Cuboid

Zhang, Chen; Hu, Yu

doi:10.20944/preprints201708.0022.v1

Cited by 7 publications

(31 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It renders the scene with the surface-splatting technique [45] and reduces the computational complexity and memory overhead compared with the volumetric approaches. The volumetric representation has been reported as difficult to resolve the highly curved and concave details such as the folds in the garment [5] or thin geometries [18] although the voxels are small enough. [18] introduced a prediction-corrected data fusion strategy for geometry details preservation.…”

Section: Related Workmentioning

confidence: 99%

“…The volumetric representation has been reported as difficult to resolve the highly curved and concave details such as the folds in the garment [5] or thin geometries [18] although the voxels are small enough. [18] introduced a prediction-corrected data fusion strategy for geometry details preservation. By storing surface normal and view ray vectors per voxel as additional information, it enables fast correction of the surface where deformation previously accumulated.…”

Section: Related Workmentioning

confidence: 99%

“…Slavcheva et al propose the SDF-2-SDF algorithm [16], [17] which focus on the small-scale object reconstruction. Similarly, we have also proposed a CuFusion framework [18] for accurate camera localization and object modelling, under the assumption that a known cuboid reference object present in the scene. A prediction-corrected TSDF fusion strategy is applied instead of simple moving average fusion, to resolve the surface deformation problem.…”

Section: Introductionmentioning

confidence: 99%

“…However, the reconstruction quality of [18] relies heavily on the quality of the raw depth measurements. To keep reconstruction fidelity in particularly highly curved zones of the objects, the input depth images should contain as less motion blur as possible, which requires the camera orbiting steadily.…”

Section: Introductionmentioning

confidence: 99%

“…Even mild blur in depth measurement caused by slightly faster camera motion may lead to reconstruction failure on sharp edges of the objects. To address these problems, in this paper we introduce a novel approach which is an extension of our previous work [18]. The major contributions are as follows:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Zhang

2018

Preprint

Self Cite

View full text Add to dashboard Cite

3D object reconstruction from depth image streams using Kinect-style depth cameras have been extensively studied. We propose an approach for accurate camera tracking and volumetric dense surface reconstruction, assuming a known cuboid reference object is present in the scene. Our contribution is threefold: (a) we keep drift-free camera pose tracking by incorporating the 3D geometric constraints of the cuboid reference object into the image registration process; (b) on the problem of depth stream fusion, we reformulate it as a binary classification problem, enabling high fidelity of surface reconstruction, especially in concave zones of the objects; (c) we further present a surface denoising strategy, facilitating the generation of  noise-free triangle mesh, making the models more suitable for 3D printing and other applications. We extend our public dataset CU3D with several fresh image sequences, test our algorithm on these sequences and compare them with other state-of-the-art algorithms. Both our dataset and algorithm are available as open-source at https://github.com/zhangxaochen/CuFusion, for other researchers to reproduce and verify our results.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Zhang

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Zhang

2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

The 3D object reconstruction from depth image streams using Kinect-style depth cameras has been extensively studied. In this paper, we propose an approach for accurate camera tracking and volumetric dense surface reconstruction, assuming that a known cuboid reference object is present in the scene. Our contribution is threefold. First, we maintain the drift-free camera pose tracking by incorporating the 3D geometric constraints of the cuboid reference object into the image registration process. Second, we reformulate the problem of depth stream fusion as a binary classification problem, enabling highfidelity surface reconstruction, especially in the concave zones of objects. Third, we further present a surface denoising strategy to mitigate the topological inconsistency (e.g., holes and dangling triangles), which facilitates the generation of a noise-free triangle mesh. We extend our public dataset CU3D with several new image sequences, test our algorithm on these sequences, and quantitatively compare them with other state-of-the-art algorithms. Both our dataset and our algorithm are available as open-source content at https://github.com/zhangxaochen/CuFusion for other researchers to reproduce and verify our results. INDEX TERMS 3D object reconstruction, depth cameras, Kinect sensors, open source, signal denoising, SLAM. CHEN ZHANG received the B.S. degree in computer science from Zhejiang University, China, in 2013, and the Ph.D. degree from the College of Computer Science and Technology, Zhejiang University, China. He is currently with the State Key Laboratory of CAD and CG, Computer Animation and Perception Group, Zhejiang University. His primary research interests include simultaneous localization and mapping (SLAM), and 3D reconstruction.

show abstract

Parallel K Nearest Neighbor Matching for 3D Reconstruction

Cao

Xie

et al. 2019

IEEE Access

View full text Add to dashboard Cite

In recent years, a 3D reconstruction based on structure from motion (SFM) has attracted much attention from the communities of computer vision and graphics. It is well known that the speed and quality of SFM systems largely depend on the technique of feature tracking. If a big volume of image data is inputted for SFM, the speed of this SFM system would become very slow. And, this problem becomes severer for largescale scenes, which typically needs to capture several thousands of images to recover the point-cloud model of the scene. However, none of the existing methods fully addresses the problem of fast feature tracking. Brute force matching is capable of producing correspondences for small-scale scenes but often getting stuck in repeated features. Hashing matching can only deal with middle-scale scenes and is not capable of largescale scenes. In this paper, we propose a new feature tacking method working in a parallel manner rather than in a single thread scheme. Our method consists of steps of keypoint detection, descriptor computing, descriptor matching by parallel k-nearest neighbor (Parallel-KNN) search, and outlier rejecting. This method is able to rapidly match a big volume of keypoints and avoids to consume high computation time, then yielding a set of correct correspondences. We demonstrate and evaluate the proposed method on several challenging benchmark datasets, including those with highly repeated features, and compare to the state-ofthe-art methods. The experimental results indicate that our method outperforms the compared methods in both efficiency and effectiveness.

show abstract

CuFusion: Accurate Real‐time Camera Tracking and Volumetric Scene Reconstruction with a Cuboid

Cited by 7 publications

References 32 publications

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Parallel K Nearest Neighbor Matching for 3D Reconstruction

Contact Info

Product

Resources

About