Multi-View Stereo Matching Based on Self-Adaptive Patch and Image Grouping for Multiple Unmanned Aerial Vehicle Imagery

Abstract

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry's requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes.(1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, applications, and prospects

“…In fact, limited progresses have been made in deep feature detection, due to the lack of large-scale annotated data and the difficulty to get a clear definition about keypoints. By contrast, great efforts have been made on developing learned descriptors based on CNNs, which have obtained (a) Matching nadir and oblique images [70] (b) Matching ground to aerial images [71] (c) Matching UAV image to geo-reference images [68] Figure 10: Low-altitude UAV image matching.…”

Studies on Three-Dimensional (3D) Modeling of UAV Oblique Imagery with the Aid of Loop-Shooting

“…A flow-chart of the incremental 3D modeling process is shown in Figure 4. After generating the sparse point cloud through 3D triangulation, the cluster multiview stereo (CMVS) [25] and patch based multiview stereo (PMVS) algorithms [26] are used to conduct cluster classification and surface calculation of the images. Then, DSM data based on the superhigh-density point clouds of real images are generated.…”

Robust Mosaicking of Lightweight UAV Images Using Hybrid Image Transformation Modeling

“…The conventional remote sensing platforms can easily meet these requirements, but lightweight UAVs might not. Because UAVs are sensitive to changes in wind direction and speed, overlaps may not be well maintained between adjacent images taken during a flight [16,17]. In addition, since UAVs have low flight altitudes, tiepoint distributions may be biased, especially with low-textured surfaces [16].…”