Integration of aerial oblique imagery and terrestrial imagery for optimized 3D modeling in urban areas

Wu, Bo; Xie, Linfu; Han, Hu; Zhu, Qing; Yau, Eric

doi:10.1016/j.isprsjprs.2018.03.004

Cited by 82 publications

(78 citation statements)

References 52 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the final step, points are segmented into two groups G n and G n−1 , whose diameters are shorter than the distance between their centroids. Max x,y∈A (d(x, y)) and Max x∈A,y∈B (d(cx, cy)) (9) where x and y are points of the same cluster and cx and cy are the centroids of cluster A and B respectively.…”

Section: Semantic Segmentationmentioning

confidence: 99%

“…SfM-based [7] methods enable the creation of geometrically precise point cloud datasets based entirely on large sets of overlapping images taken from multiple viewpoints. Recent research has been presented for the 3D reconstruction of urban and natural environments by using conventional cameras [8,9]. Instead of managing a triangular mesh, which requires an additional computational cost, the point clouds are widely used to provide an accurate representation of existing objects in nature.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features

Jurado

Cárdenas

Ogáyar

et al. 2020

Sensors

View full text Add to dashboard Cite

The characterization of natural spaces by the precise observation of their material properties is highly demanded in remote sensing and computer vision. The production of novel sensors enables the collection of heterogeneous data to get a comprehensive knowledge of the living and non-living entities in the ecosystem. The high resolution of consumer-grade RGB cameras is frequently used for the geometric reconstruction of many types of environments. Nevertheless, the understanding of natural spaces is still challenging. The automatic segmentation of homogeneous materials in nature is a complex task because there are many overlapping structures and an indirect illumination, so the object recognition is difficult. In this paper, we propose a method based on fusing spatial and multispectral characteristics for the unsupervised classification of natural materials in a point cloud. A high-resolution camera and a multispectral sensor are mounted on a custom camera rig in order to simultaneously capture RGB and multispectral images. Our method is tested in a controlled scenario, where different natural objects coexist. Initially, the input RGB images are processed to generate a point cloud by applying the structure-from-motion (SfM) algorithm. Then, the multispectral images are mapped on the three-dimensional model to characterize the geometry with the reflectance captured from four narrow bands (green, red, red-edge and near-infrared). The reflectance, the visible colour and the spatial component are combined to extract key differences among all existing materials. For this purpose, a hierarchical cluster analysis is applied to pool the point cloud and identify the feature pattern for every material. As a result, the tree trunk, the leaves, different species of low plants, the ground and rocks can be clearly recognized in the scene. These results demonstrate the feasibility to perform a semantic segmentation by considering multispectral and spatial features with an unknown number of clusters to be detected on the point cloud. Moreover, our solution is compared to other method based on supervised learning in order to test the improvement of the proposed approach.

show abstract

Section: Semantic Segmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features

Jurado

Cárdenas

Ogáyar

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Merging oblique aerial images, as well as terrestrial images, for creating complete representations of a scene has been pursued by Wu et al. (). Although this is a different task with the image data already co‐registered with respect to their orientation, a method similar to the one used in the Wu et al.…”

Section: Related Workmentioning

confidence: 99%

“…These approaches only utilise vertical aerial images, and thus have similar limitations to the current authors' previous approach as far as the availability of potential correspondences is concerned. Merging oblique aerial images, as well as terrestrial images, for creating complete representations of a scene has been pursued by Wu et al (2018). Although this is a different task with the image data already co-registered with respect to their orientation, a method similar to the one used in the Wu et al paper is used, as those authors relied on plane priors extracted from 3D meshes.…”

Section: Related Workmentioning

confidence: 99%

Co‐registration of panoramic mobile mapping images and oblique aerial images

Fanta-Jende

Nex

Vosselman

et al. 2019

The Photogrammetric Record

View full text Add to dashboard Cite

Mobile mapping relies on satellite‐based positioning, which suffers from line‐of‐sight and multipath issues. As an alternative, this paper presents a fully automatic approach for the co‐registration of mobile mapping and oblique aerial images to introduce highly accurate and reliable ground control for mobile mapping data adjustment. An oblique view of a scene introduces similarities as well as challenges regarding co‐registration with mobile mapping images, which is supported by mutual planes in both datasets. Façade planes from a sparse point cloud are used as projection surfaces for the mobile mapping and aerial datasets, overcoming large perspective differences between them to simplify the registration. The performance of the procedure indicates an inlier rate of around 80%.

show abstract

“…If the orientation of the image pair is coarsely known, the registration task can be facilitated considerably. Wu et al (2018) suggest registering terrestrial and aerial oblique images with the support of plane priors extracted from 3D meshes. Their aim is to combine terrestrial and aerial point clouds to achieve a complementary coverage as well as to increase the resolution of the resulting 3D model.…”

Section: Related Workmentioning

confidence: 99%

A Guided Registration Strategy Employing Virtual Planes to Overcome Non-Standard Geometries – Using the Example of Mobile Mapping and Aerial Oblique Imagery

Jende

Nex

Gerke

et al. 2018

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

ABSTRACT:Mobile mapping (MM) is an intriguing as well as emerging platform and technology for geo-data acquisition. In typical areas of interest for MM campaigns, such as urban areas, unwanted GNSS multipath, non-line-of-sight effects, and IMU drifts may lead to deteriorated position fixes. In this work, we are proposing a novel technique to register MM and aerial oblique imagery. As aerial platforms are not affected by GNSS occlusions and are able to collect very-high resolution images, a co-registration of the data sets enables a) an independent verification of the platform's accuracy and b) an adjustment of the MM data's pose. Both data sets depict the scene from an entirely different perspective, which complicates the matching problem. Our approach is based on the assumption that common visible entities in both images are available, e.g. façade surfaces. By determining planes coinciding with these visible entities in object space, variances can be overcome. As the orientation of the data sets is known -MM data has an unknown accuracy -derived planes are employed to support a visibility hypothesis while storing image information for image registration in object space. This enables constraining search space to support the actual registration. Although the inhomogeneity of the data sets poses a challenge to a successful registration, we can show that our stepwise strategy of finding common visible entities, exploiting them to increase the resemblance of the data sets, and utilising accurate registration methods renders this matching scenario possible. In this paper, the algorithm is explained in detail, experimental results of significant steps will be shown, and possible extensions are discussed.

show abstract

Integration of aerial oblique imagery and terrestrial imagery for optimized 3D modeling in urban areas

Cited by 82 publications

References 52 publications

Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features

Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features

Co‐registration of panoramic mobile mapping images and oblique aerial images

A Guided Registration Strategy Employing Virtual Planes to Overcome Non-Standard Geometries – Using the Example of Mobile Mapping and Aerial Oblique Imagery

Contact Info

Product

Resources

About