Real-time Seamless Image Stitching in Autonomous Driving

Kinzig, Christian; Cortes, Irene; Fernández, Carlos; Lauer, Martin

doi:10.23919/fusion49751.2022.9841307

Cited by 10 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image stitching plays a critical role in a diverse array of applications ranging from autonomous vehicle driving and navigation [ 61 , 62 ] to augmented reality [ 63 ], medical imaging [ 64 , 65 ] and vehicle exterior monitoring [ 29 ]. Traditionally, image stitching involves aligning and merging multiple images taken from different perspectives of the same point into a seamless, high-resolution panorama or mosaic.…”

Section: Related Work and Backgroundmentioning

confidence: 99%

A Novel Framework for Image Matching and Stitching for Moving Car Inspection under Illumination Challenges

El Saer,

Grammatikopoulos,

Sfikas

et al. 2024

Sensors

View full text Add to dashboard Cite

Vehicle exterior inspection is a critical operation for identifying defects and ensuring the overall safety and integrity of vehicles. Visual-based inspection of moving objects, such as vehicles within dynamic environments abounding with reflections, presents significant challenges, especially when time and accuracy are of paramount importance. Conventional exterior inspections of vehicles require substantial labor, which is both costly and prone to errors. Recent advancements in deep learning have reduced labor work by enabling the use of segmentation algorithms for defect detection and description based on simple RGB camera acquisitions. Nonetheless, these processes struggle with issues of image orientation leading to difficulties in accurately differentiating between detected defects. This results in numerous false positives and additional labor effort. Estimating image poses enables precise localization of vehicle damages within a unified 3D reference system, following initial detections in the 2D imagery. A primary challenge in this field is the extraction of distinctive features and the establishment of accurate correspondences between them, a task that typical image matching techniques struggle to address for highly reflective moving objects. In this study, we introduce an innovative end-to-end pipeline tailored for efficient image matching and stitching, specifically addressing the challenges posed by moving objects in static uncalibrated camera setups. Extracting features from moving objects with strong reflections presents significant difficulties, beyond the capabilities of current image matching algorithms. To tackle this, we introduce a novel filtering scheme that can be applied to every image matching process, provided that the input features are sufficient. A critical aspect of this module involves the exclusion of points located in the background, effectively distinguishing them from points that pertain to the vehicle itself. This is essential for accurate feature extraction and subsequent analysis. Finally, we generate a high-quality image mosaic by employing a series of sequential stereo-rectified pairs.

show abstract

Section: Related Work and Backgroundmentioning

confidence: 99%

A Novel Framework for Image Matching and Stitching for Moving Car Inspection under Illumination Challenges

El Saer,

Grammatikopoulos,

Sfikas

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…With the rapid development of image stitching and image fusion technologies, methods for obtaining multi-view or even global perspectives through multiple single viewpoints have been widely applied in human production and daily life [1][2][3][4][5]. For instance, the extensive use of technologies such as panoramic images, autonomous driving, and virtual reality (VR) enables the precise remote observation of scenes by individuals [6][7][8]. However, in the process of stitching multiple single-view images, it is necessary to align the overlapping regions of different images by adjusting their positions, angles, and local distortions [9].…”

Section: Introductionmentioning

confidence: 99%

Rectification for Stitched Images with Deformable Meshes and Residual Networks

Fan,

Mao,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Image stitching is an important method for digital image processing, which is often prone to the problem of the irregularity of stitched images after stitching. And the traditional image cropping or complementation methods usually lead to a large number of information loss. Therefore, this paper proposes an image rectification method based on deformable mesh and residual network. The method aims to minimize the information loss at the edges of the spliced image and the information loss inside the image. Specifically, the method can select the most suitable mesh shape for residual network regression according to different images. Its loss function includes global loss and local loss, aiming to minimize the loss of image information within the grid and global target. The method in this paper not only greatly reduces the information loss caused by irregular shapes after image stitching, but also adapts to different images with various rigid structures. Meanwhile, its validation on the DIR-D dataset shows that the method outperforms the state-of-the-art methods in image rectification.

show abstract

“…With the rapid development of image stitching and image fusion technologies, methods for obtaining multi-view or even global perspectives through multiple single viewpoints have been widely applied in human production and daily life [1][2][3][4][5]. For instance, the extensive use of technologies such as panoramic images, autonomous driving, and virtual reality (VR) enables precise remote observation of scenes by individuals [6][7][8][9]. However, in the process of stitching multiple single-view images, it is necessary to align the overlapping regions of different images by adjusting their positions, angles, and local distortions [10,11].…”

Section: Introductionmentioning

confidence: 99%

Rectification for Image Stitching with Deformable Meshes and Residual Networks

Fan,

Mao,

Mei

et al. 2024

Preprint

View full text Add to dashboard Cite

Image stitching is a crucial aspect of image processing. However, factors like perspective and environment often lead to irregular shapes in stitched images. Cropping or completion methods typically result in substantial loss of information. This paper proposes a method for rectifying irregularly images into rectangles using deformable meshes and residual networks. The method utilizes a convolutional neural network to quantify rigid structures of images. Choosing the most suitable mesh structure based on the extraction results, offering options such as triangular, rectangular, and hexagonal. Subsequently, the irregularly image, predefined mesh structure, and predicted mesh structure are input into a wide residual neural network for regression. The loss function comprises local and global, aimed at minimizing the loss of image information within the mesh and global target. This method not only significantly reduces information loss during rectification but also adapting to different images with various rigid structures. Validation on the DIR-D dataset shows this method outperforms state-of-the-art methods in image rectification.

show abstract

Real-time Seamless Image Stitching in Autonomous Driving

Cited by 10 publications

References 18 publications

A Novel Framework for Image Matching and Stitching for Moving Car Inspection under Illumination Challenges

A Novel Framework for Image Matching and Stitching for Moving Car Inspection under Illumination Challenges

Rectification for Stitched Images with Deformable Meshes and Residual Networks

Rectification for Image Stitching with Deformable Meshes and Residual Networks

Contact Info

Product

Resources

About