Semantic Photogrammetry – Boosting Image-Based 3d Reconstruction With Semantic Labeling

Stathopoulou, E. K.; Remondino, Fabio

doi:10.5194/isprs-archives-xlii-2-w9-685-2019

Cited by 30 publications

(30 citation statements)

References 32 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite some works that attempt at classifying DCH images by employing different kinds of techniques [33][34][35][36] already exist, there are still few researches who seek to directly exploit the Point Clouds of CH for semantic classification or segmentation through ML [37] or DL techniques. One of them is [38], where a segmentation of 3D models of historical buildings is proposed for FEA analysis, starting from Point Clouds and meshes.…”

Section: Classification and Semantic Segmentation In The Field Of Dchmentioning

confidence: 99%

Point Cloud Semantic Segmentation Using a Deep Learning Framework for Cultural Heritage

et al. 2020

View full text Add to dashboard Cite

In the Digital Cultural Heritage (DCH) domain, the semantic segmentation of 3D Point Clouds with Deep Learning (DL) techniques can help to recognize historical architectural elements, at an adequate level of detail, and thus speed up the process of modeling of historical buildings for developing BIM models from survey data, referred to as HBIM (Historical Building Information Modeling). In this paper, we propose a DL framework for Point Cloud segmentation, which employs an improved DGCNN (Dynamic Graph Convolutional Neural Network) by adding meaningful features such as normal and colour. The approach has been applied to a newly collected DCH Dataset which is publicy available: ArCH (Architectural Cultural Heritage) Dataset. This dataset comprises 11 labeled points clouds, derived from the union of several single scans or from the integration of the latter with photogrammetric surveys. The involved scenes are both indoor and outdoor, with churches, chapels, cloisters, porticoes and loggias covered by a variety of vaults and beared by many different types of columns. They belong to different historical periods and different styles, in order to make the dataset the least possible uniform and homogeneous (in the repetition of the architectural elements) and the results as general as possible. The experiments yield high accuracy, demonstrating the effectiveness and suitability of the proposed approach.

show abstract

Section: Classification and Semantic Segmentation In The Field Of Dchmentioning

confidence: 99%

Point Cloud Semantic Segmentation Using a Deep Learning Framework for Cultural Heritage

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Chen, Y. et al [7] and Stathopoulou, E.K. et al [23] filter the mismatching by semantic labels of feature points. With the motivation of denoising, Zhang, R. et al [24] proposed a Hough-transform-based algorithm called FC-GHT to detect plane on point cloud for further semantic label optimization.…”

Section: Related Workmentioning

confidence: 99%

“…Stathopoulou, E.K. et al [23] used semantic information as a mask to wipe out the meshes belonging to the semantic class sky. These methods have two primary drawbacks.…”

Section: Related Workmentioning

confidence: 99%

Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images

Wei

Wang

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

Semantic modeling is a challenging task that has received widespread attention in recent years. With the help of mini Unmanned Aerial Vehicles (UAVs), multi-view high-resolution aerial images of large-scale scenes can be conveniently collected. In this paper, we propose a semantic Multi-View Stereo (MVS) method to reconstruct 3D semantic models from 2D images. Firstly, 2D semantic probability distribution is obtained by Convolutional Neural Network (CNN). Secondly, the calibrated cameras poses are determined by Structure from Motion (SfM), while the depth maps are estimated by learning MVS. Combining 2D segmentation and 3D geometry information, dense point clouds with semantic labels are generated by a probability-based semantic fusion method. In the final stage, the coarse 3D semantic point cloud is optimized by both local and global refinements. By making full use of the multi-view consistency, the proposed method efficiently produces a fine-level 3D semantic point cloud. The experimental result evaluated by re-projection maps achieves 88.4% Pixel Accuracy on the Urban Drone Dataset (UDD). In conclusion, our graph-based semantic fusion procedure and refinement based on local and global information can suppress and reduce the re-projection error.

show abstract

“…The proposed semantic photogrammetry reconstruction pipeline where semantic priors are incorporated to support the 3D results (modified fromStathopoulou and Remondino, 2019).…”

mentioning

confidence: 99%

“…Input set of images and the network orientation with the resulting sparse point cloud. Image labels are projected to 3D, producing a semantically segmented dense point cloud, as obtained by the pipeline presented inStathopoulou and Remondino (2019).…”

mentioning

confidence: 99%

Multi-View Stereo With Semantic Priors

Stathopoulou

Remondino

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

<p><strong>Abstract.</strong> Patch-based stereo is nowadays a commonly used image-based technique for dense 3D reconstruction in large scale multi-view applications. The typical steps of such a pipeline can be summarized in stereo pair selection, depth map computation, depth map refinement and, finally, fusion in order to generate a complete and accurate representation of the scene in 3D. In this study, we aim to support the standard dense 3D reconstruction of scenes as implemented in the open source library OpenMVS by using semantic priors. To this end, during the depth map fusion step, along with the depth consistency check between depth maps of neighbouring views referring to the same part of the 3D scene, we impose extra semantic constraints in order to remove possible errors and selectively obtain segmented point clouds per label, boosting automation towards this direction. In order to reassure semantic coherence between neighbouring views, additional semantic criterions can be considered, aiming to eliminate mismatches of pixels belonging in different classes.</p>

show abstract

Semantic Photogrammetry – Boosting Image-Based 3d Reconstruction With Semantic Labeling

Cited by 30 publications

References 32 publications

Point Cloud Semantic Segmentation Using a Deep Learning Framework for Cultural Heritage

Point Cloud Semantic Segmentation Using a Deep Learning Framework for Cultural Heritage

Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images

Multi-View Stereo With Semantic Priors

Contact Info

Product

Resources

About