Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han, Xuefei; Laga, Hamid; Bennamoun, Mohammed

doi:10.1109/tpami.2019.2954885

Cited by 272 publications

(136 citation statements)

References 147 publications

(317 reference statements)

Supporting

Mentioning

134

Contrasting

Unclassified

Order By: Relevance

“…Image-based 3D reconstruction technology [3] provides an efficient and low-cost solution to generate 3D models only with a set of photos from multiple viewpoints. Although such reconstruction technology is less accurate compared to some laser scanning based technologies (e.g., simultaneous localization and mapping (SLAM) [4]), it is easier to be popularized with much less effort and cost.…”

Section: Real-time 3d Reconstructionmentioning

confidence: 99%

“…Generally, these applications require accurate, large-scale, and dense 3D models of environments in (or close to) realtime, thus are delay intensive and resource hungry. 3D reconstruction [3], [4], [5] is a very well studied problem in computer vision. Traditional image-based reconstruction techniques [5], [6] require a mobile user to capture multiple RGB images of the same object from multiple viewpoints.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Edge Computing-Based Photo Crowdsourcing Framework for Real-Time 3D Reconstruction

Chen

Wang

et al. 2022

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

Image-based three-dimensional (3D) reconstruction utilizes a set of photos to build 3D model and can be widely used in many emerging applications such as augmented reality (AR) and disaster recovery. Most of existing 3D reconstruction methods require a mobile user to walk around the target area and reconstruct objectives with a hand-held camera, which is inefficient and time-consuming. To meet the requirements of delay intensive and resource hungry applications in 5G, we propose an edge computing-based photo crowdsourcing (EC-PCS) framework in this paper. The main objective is to collect a set of representative photos from ubiquitous mobile and Internet of Things (IoT) devices at the network edge for real-time 3D model reconstruction, with network resource and monetary cost considerations. Specifically, we first propose a photo pricing mechanism by jointly considering their freshness, resolution and data size. Then, we design a novel photo selection scheme to dynamically select a set of photos with the required target coverage and the minimum monetary cost. We prove the NP-hardness of such problem, and develop an efficient greedy-based approximation algorithm to obtain a near-optimal solution. Moreover, an optimal network resource allocation scheme is presented, in order to minimize the maximum uploading delay of the selected photos to the edge server. Finally, a 3D reconstruction algorithm and a 3D model caching scheme are performed by the edge server in real time. Extensive experimental results based on real-world datasets demonstrate the superior performance of our EC-PCS system over the existing mechanisms. Index Terms-three-dimensional (3D) reconstruction, multi-access edge computing, photo crowdsourcing.

show abstract

Section: Real-time 3d Reconstructionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Edge Computing-Based Photo Crowdsourcing Framework for Real-Time 3D Reconstruction

Chen

Wang

et al. 2022

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

show abstract

“…A 3D-CNN employs 3D convolutions in its convolutional layers. 3D-CNNs are popular for video classification [36], 3D object reconstruction [56], and action recognition [57]. For the case of HSIs, the form of the 3D filter suits the data structure of the HSI cube.…”

Section: Cnnmentioning

confidence: 99%

A High-Performance Spectral-Spatial Residual Network for Hyperspectral Image Classification with Small Training Data

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a high performance Two-Stream spectral-spatial Residual Network (TSRN) for hyperspectral image classification. The first spectral residual network (sRN) stream is used to extract spectral characteristics, and the second spatial residual network (saRN) stream is concurrently used to extract spatial features. The sRN uses 1D convolutional layers to fit the spectral data structure, while the saRN uses 2D convolutional layers to match the hyperspectral spatial data structure. Furthermore, each convolutional layer is preceded by a Batch Normalization (BN) layer that works as a regularizer to speed up the training process and to improve the accuracy. We conducted experiments on three well-known hyperspectral datasets, and we compare our results with five contemporary methods across various sizes of training samples. The experimental results show that the proposed architecture can be trained with small size datasets and outperforms the state-of-the-art methods in terms of the Overall Accuracy, Average Accuracy, Kappa Value, and training time.

show abstract

“…Since 2015, learning-based methods for surface reconstruction have received much attention. However, at present, such techniques are mainly focused on images and objects spanned by the training set [30,31] Table 1). After the study, the authors concluded that Ball Pivoting and Alpha Shapes generate geometrically precise reconstructions of the input data on a global scale, but are sensitive to noise, have long runtimes and create topologically inconsistent meshes with locally undesirable triangulations.…”

Section: Triangulation/mesh-based Algorithmsmentioning

confidence: 99%

Multisource Point Clouds, Point Simplification and Surface Reconstruction

et al. 2019

View full text Add to dashboard Cite

As data acquisition technology continues to advance, the improvement and upgrade of the algorithms for surface reconstruction are required. In this paper, we utilized multiple terrestrial Light Detection And Ranging (Lidar) systems to acquire point clouds with different levels of complexity, namely dynamic and rigid targets for surface reconstruction. We propose a robust and effective method to obtain simplified and uniform resample points for surface reconstruction. The method was evaluated. A point reduction of up to 99.371% with a standard deviation of 0.2 cm was achieved. In addition, well-known surface reconstruction methods, i.e., Alpha shapes, Screened Poisson reconstruction (SPR), the Crust, and Algebraic point set surfaces (APSS Marching Cubes), were utilized for object reconstruction. We evaluated the benefits in exploiting simplified and uniform points, as well as different density points, for surface reconstruction. These reconstruction methods and their capacities in handling data imperfections were analyzed and discussed. The findings are that i) the capacity of surface reconstruction in dealing with diverse objects needs to be improved; ii) when the number of points reaches the level of millions (e.g., approximately five million points in our data), point simplification is necessary, as otherwise, the reconstruction methods might fail; iii) for some reconstruction methods, the number of input points is proportional to the number of output meshes; but a few methods are in the opposite; iv) all reconstruction methods are beneficial from the reduction of running time; and v) a balance between the geometric details and the level of smoothing is needed. Some methods produce detailed and accurate geometry, but their capacity to deal with data imperfection is poor, while some other methods exhibit the opposite characteristics.

show abstract

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Cited by 272 publications

References 147 publications

An Edge Computing-Based Photo Crowdsourcing Framework for Real-Time 3D Reconstruction

An Edge Computing-Based Photo Crowdsourcing Framework for Real-Time 3D Reconstruction

A High-Performance Spectral-Spatial Residual Network for Hyperspectral Image Classification with Small Training Data

Multisource Point Clouds, Point Simplification and Surface Reconstruction

Contact Info

Product

Resources

About