A Novel Projective-Consistent Plane Based Image Stitching Method

Zheng, Jin; Wang, Yue; Wang, Hanzi; Li, Bo; Hu, Hai‐Miao

doi:10.1109/tmm.2019.2905692

Cited by 37 publications

(24 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lee et al [7] proposed a new concept of the warping residual, which is based on a warping residual to stitch large parallax images. Zheng et al [8] used the normal vector's orientations of local regions and the projection errors of aligned images to segment an image; to improve the image registration accuracy. Li et al [9] proposed an image stitching method based on robust elastic warping, which can stitch large parallax images and achieve accurate registration.…”

Section: Uav Remote Sensing Image Stitchingmentioning

confidence: 99%

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

2021

View full text Add to dashboard Cite

Ghosts are a common phenomenon widely present in unmanned aerial vehicle (UAV) remote sensing image stitching that seriously affect the naturalness of stitching results. In order to effectively remove ghosts and produce visually natural stitching results, we propose a novel image stitching method that can identify and eliminate ghosts through multi-component collaboration without object distortion, segmentation or repetition. Specifically, our main contributions are as follows: first, we propose a ghost identification component to locate a potential ghost in the stitching area; and detect significantly moving objects in the two stitched images. In particular, due to the characteristics of UAV shooting, the objects in UAV remote sensing images are small and the image quality is poor. We propose a mesh-based image difference comparison method to identify ghosts; and use an object tracking algorithm to accurately correspond to each ghost pair. Second, we design an image information source selection strategy to generate the ghost replacement region, which can replace the located ghost and avoid object distortion, segmentation and repetition. Third, we find that the process of ghost elimination can produce natural mosaic images by eliminating the ghost caused by initial blending with selected image information source. We validate the proposed method on VIVID data set and compare our method with Homo, ELA, SPW and APAP using the peak signal to noise ratio (PSNR) evaluation indicator.

show abstract

Section: Uav Remote Sensing Image Stitchingmentioning

confidence: 99%

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

2021

View full text Add to dashboard Cite

show abstract

“…For plane segmentation, Lin et al [6] segmented an image roughly into the distant plane and the ground plane through clustering on keypoints and weighted averaging. Zheng et al [29] used detected keypoints as vertexes of a triangulated mesh from which a plane segmentation is formulated. Liu et al [20] designed PlaneNet to solve plane detection and parameter and depth estimation from a single RGB image under a united framework.…”

Section: Related Workmentioning

confidence: 99%

Image Stitching Based on Semantic Planar Region Consensus

Guo

2021

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Image stitching for two images without a global transformation between them is notoriously difficult. In this paper, noticing the importance of planar structure under perspective geometry, we propose a new image stitching method which stitches images by allowing for the alignment of a set of matched dominant planar regions. Clearly different from previous methods resorting to plane segmentation, the key to our approach is to utilize rich semantic information directly from RGB images to extract planar image regions with a deep Convolutional Neural Network (CNN). We specifically design a new module to make fully use of existing semantic segmentation networks to accommodate planar segmentation. To train the network, a dataset for planar region segmentation is contributed. With the planar region knowledge, a set of local transformations can be obtained by constraining matched regions, enabling more precise alignment in the overlapping area. We also use planar knowledge to estimate a transformation field over the whole image. The final mosaic is obtained by a mesh-based optimization framework which maintains high alignment accuracy and relaxes similarity transformation at the same time. Extensive experiments with quantitative comparisons show that our method can deal with different situations and outperforms the state-of-the-arts on challenging scenes.

show abstract

“…Image stitching has been extensively studied, and commercial software is available for various camera systems. One of the major challenges in image stitching is in correcting the parallax error [1,2]. A parallax error occurs when the non-parallax point of a camera is moved while capturing a three-dimensional (3D) scene, resulting in image misalignments, such as a ghost artifact.…”

Section: Introductionmentioning

confidence: 99%

Quantitative Assessment Method of Image Stitching Performance Based on Estimation of Planar Parallax

Jung

Hong

2021

IEEE Access

View full text Add to dashboard Cite

While parallax-tolerant image stitching is a relatively mature field, the performances of image stitching methods have been assessed subjectively and qualitatively. These methods primarily provide the stitched image itself to demonstrate the performance, rather than quantitative data. Although several objective assessment methods have been proposed for quantifying the quality of stitched images, only the stitched output images have been analyzed, without considering the parallax level in each input image. We propose a method for quantifying the parallax level of the input images and clustering them accordingly. This facilitates a quantitative assessment of the various stitching methods for each parallax level. The parallax levels of the images are grouped based on the magnitude and variation in the planar parallax, as estimated with the proposed metric using matching errors and patch similarity. The existing image stitching methods are compared experimentally in terms of the residual misalignment errors, based on 73 pairs of different levels of parallax images originally classified in this study. Among the existing methods, the elastic local alignment method exhibits the least error. The shape-preserving half-projective method produces a larger misalignment error, but creates a natural panorama with less geometric distortion. We introduce a quantitative assessment method for considering the parallax of input images in image stitching methods. It can aid in specifying their performances, and in finding an appropriate method depending on the parallax level of the input images.INDEX TERMS Image alignment, image stitching, planar parallax, plane + parallax, quantitative assessment.

show abstract

A Novel Projective-Consistent Plane Based Image Stitching Method

Cited by 37 publications

References 32 publications

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

Image Stitching Based on Semantic Planar Region Consensus

Quantitative Assessment Method of Image Stitching Performance Based on Estimation of Planar Parallax

Contact Info

Product

Resources

About