3D-FHNet: Three-Dimensional Fusion Hierarchical Reconstruction Method for Any Number of Views

Deep learning‐based single view 3D reconstruction is a hot topic in computer vision. However, predicting a more realistic 3D point cloud from a single image is an ill‐posed problem. In recent years, most of the 3D point cloud prediction methods based on single view are straight‐through structure, which will cause the loss of part of feature information and the loss of part of detail information of the resulting point clouds, which will lead to the unsatisfactory visual effect of reconstructed point clouds. In this paper, a Feature‐Enhanced 3D point clouds generation Network (3D‐FENet) from a single image is proposed. In order to enhance the feature information of RGB image, edge extraction module is adopted. In the process of point cloud generation, a point cloud pyramid is designed, which combines low resolution point cloud with high resolution point cloud to enhance the local details of the generated point clouds. In the fine‐tuning stage, the differential projection module is used to fine‐tune the whole network by 2D projection of reconstructed point clouds. Experimental results show that the performance of the authors’ proposed method is better than the state‐of‐the‐art studies.

show abstract

“…Lu et al. [20] proposed a method that can encode any number of views through LSTM to generate 3D voxels, named 3D‐FHNet. Choy et al.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, the TL-embedding Network is proposed to map 2D image to 3D voxel, but the resolution is low. Lu et al [20] proposed a method that can encode any number of views through LSTM to generate 3D voxels, named 3D-FHNet. Choy et al [12] designed a RNN to map multipleview 2D images to 3D voxels named 3D-R2N2.…”

Section: Related Workmentioning

confidence: 99%

3D‐FEGNet: A feature enhanced point cloud generation network from a single image

Wang

Hui

Wang

et al. 2022

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…3D-R2N2 [6] proposed to use long short-term memory (LSTM) to infer the 3D geometry using many images of the target object from different perspectives. Recently, 3D-FHNet, which is a 3D fusion Hierarchical reconstruction method, was proposed that can perform 3D object reconstruction of any number of views [14]. The critical limitation of using the volumetric representation in the above-mentioned methods is the computational and the memory cost and the restriction on the output resolution.…”

Section: Related Workmentioning

confidence: 99%

Pixel2point: 3D Object Reconstruction From a Single Image Using CNN and Initial Sphere

et al. 2021

View full text Add to dashboard Cite

“…Recent works combine Deep Learning techniques to perform three-dimensional reconstructions, using data from the stereo cameras, mono-LiDAR, and stereo-LiDAR cameras, merging merging the data from these sensors to obtain better results to obtain better results [35][36][37][38][39]. However, these proposals are focused on solutions for individual objects and many of them only focus on reconstructing structured environments.…”

Section: Related Workmentioning

confidence: 99%

A Deep Learning Architecture for 3D Mapping Urban Landscapes

et al. 2021

View full text Add to dashboard Cite

In this paper, an approach through a Deep Learning architecture for the three-dimensional reconstruction of outdoor environments in challenging terrain conditions is presented. The architecture proposed is configured as an Autoencoder. However, instead of the typical convolutional layers, some differences are proposed. The Encoder stage is set as a residual net with four residual blocks, which have been provided with the necessary knowledge to extract the feature maps from aerial images of outdoor environments. On the other hand, the Decoder stage is set as a Generative Adversarial Network (GAN) and called a GAN-Decoder. The proposed network architecture uses a sequence of the 2D aerial image as input. The Encoder stage works for the extraction of the vector of features that describe the input image, while the GAN-Decoder generates a point cloud based on the information obtained in the previous stage. By supplying a sequence of frames that a percentage of overlap between them, it is possible to determine the spatial location of each generated point. The experiments show that with this proposal it is possible to perform a 3D representation of an area flown over by a drone using the point cloud generated with a deep architecture that has a sequence of aerial 2D images as input. In comparison with other works, our proposed system is capable of performing three-dimensional reconstructions in challenging urban landscapes. Compared with the results obtained using commercial software, our proposal was able to generate reconstructions in less processing time, with less overlapping percentage between 2D images and is invariant to the type of flight path.

show abstract

3D-FHNet: Three-Dimensional Fusion Hierarchical Reconstruction Method for Any Number of Views

Cited by 9 publications

References 29 publications

3D‐FEGNet: A feature enhanced point cloud generation network from a single image

3D‐FEGNet: A feature enhanced point cloud generation network from a single image

Pixel2point: 3D Object Reconstruction From a Single Image Using CNN and Initial Sphere

A Deep Learning Architecture for 3D Mapping Urban Landscapes

Contact Info

Product

Resources

About