Modeling realistic optical aberrations to reuse existing drive scene recordings for autonomous driving validation

Lehmann, Matthias; Wittpahl, Christian; Zakour, Hatem Ben; Braun, Alexander

doi:10.1117/1.jei.28.1.013005

Cited by 4 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The PSF describes effects such as diffraction phenomena and lens aberrations and is considered a non-linear and space-variant function. [19,15,14]. Often the space-variance is neglected, if it can be assured that details of the introduced blur don't matter.…”

Section: Optical Model and Filteringmentioning

confidence: 99%

“…We have developed novel degradation models that transform a given image as if recorded by a different lens with different optical properties. [15,14,17,13] This lens is applied like a filtering process on a given image database, and thus allows for the parameterized simulation of different optical properties. This work examines the influence of our optical models on the noise present in prerecorded scenes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Original image noise reconstruction for spatially-varying filtered driving scenes

Constantin¹,

Maillard²,

Braun³

2022

Self Cite

View full text Add to dashboard Cite

Test drives for the development of camera-based automotive algorithms like object detection or instance segmentation are very expensive and time-consuming. Therefore, the re-use of existing databases like COCO or Berkeley Deep Drive by intentionally varying the image quality in a post-processing step promises to save time and money, while giving access to novel image quality properties. One possible variation we investigate is the sharpness of the camera system, by applying spatially varying optical blur models as low-pass filters on the image data. Any such operation significantly changes the amount and distribution of noise, a central property of image quality, which in this context is an undesired side-effect. In this article, a novel method is presented to reconstruct the original camera sensor noise for the filtered image. This is different from denoising. The method estimates the original camera sensor noise using the combination of principal component analysis (PCA) and a variance-stabilizing transformation. The noise is then reconstructed for the filtered image with the PCA applied locally on small image sections, and an inverse variance-stabilizing transformation. Although the resulting noise distribution can slightly deviate from the original, this novel method does not introduce any image artifacts as denoising would do. We present the method as applied to synthetic and real driving scenes at different noise levels and discuss the accuracy of the reconstruction visually and with statistical parameters.

show abstract

Section: Optical Model and Filteringmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Original image noise reconstruction for spatially-varying filtered driving scenes

Constantin¹,

Maillard²,

Braun³

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our group we focus our research on the physical-realistic simulation of images for the training of algorithms used in an au-tomotive context, for advanced driver assistance systems (ADAS) or autonomous driving (AD). Using realistic optical models such as [7,6] or [5] we degrade given images -both from simulation and already recorded drive scenes -to investigate the effect that the different optical properties have on CV algorithms [9,5]. Realistic optical properties vary over the field of view, they are not translation-invariant.…”

Section: Introductionmentioning

confidence: 99%

Spatial precision and recall indices to assess the performance of instance segmentation algorithms

Mattis¹,

Maillard²,

Braun³

2022

Self Cite

View full text Add to dashboard Cite

Since it is essential for Computer Vision systems to reliably perform in safety-critical applications such as autonomous vehicles, there is a need to evaluate their robustness to naturally occurring image perturbations. More specifically, the performance of Computer Vision systems needs to be linked to the image quality, which hasn't received much research attention so far. In fact, aberrations of a camera system are always spatially variable over the Field of View, which may influence the performance of Computer Vision systems dependent on the degree of local aberrations. Therefore, the goal is to evaluate the performance of Computer Vision systems under effects of defocus by taking into account the spatial domain. Large-scale Autonomous Driving datasets are degraded by a parameterized optical model to simulate driving scenes under physically realistic effects of defocus. Using standard evaluation metrics, the Spatial Recall Index (SRI) and the new Spatial Precision Index (SPI), the performance of Computer Visions systems on these degraded datasets are compared with the optical performance of the applied optical model. A correlation could be observed between the spatially varying optical performance and the spatial performance of Instance Segmentation systems.

show abstract

“…The concept of synthesizing digital images is the main focus of computer graphics. Considering the huge recent success of rendering methods in generating highly realistic images for film and game productions, rendering techniques have found rather new use cases, such as generating training data for machine learning tasks [6] or simulating realistic images for training autonomous vehicles [7]. The idea of utilizing simulations for evaluation of optical inspection systems was first introduced by Retzlaff et al [8].…”

Section: Introductionmentioning

confidence: 99%

Sensor-Realistic Simulations for Evaluation and Planning of Optical Measurement Systems With an Application to Laser Triangulation

et al. 2020

View full text Add to dashboard Cite

Image acquisition plays a central role in optical measurement systems. To configure machine vision setups, engineers often go through some empirical best practices to evaluate the measurement in different geometrical and optical configurations. This approach turns out to be expensive, tedious, and sometimes even unsuitable for nontrivial tasks. To automate this process, realistic synthetic images allow us to evaluate and optimize imaging setups without employing physical parts and sensors. Available computer graphics rendering techniques can be very beneficial for realistic simulation of images in machine vision systems. Simulating optical measurement systems, however, requires taking extra simulation components into account, including the wave optics effects and realistic spectral response and stochastic noise of the digital sensor. In this article, we focus on physical simulation of optical measurement systems and further propose and verify a sensor-realistic simulation framework for this purpose. We demonstrate the results in different steps for a laser triangulation measurement system, and verify them against experimental images, both qualitatively and quantitatively. The measurement with a laser light source results in wave optics effects which cannot be modeled by conventional ray-tracing methods. The concepts of this article are, however, general to many optical systems with both coherent and/or incoherent light sources. The agreement of simulations with reality is a promising proof of concept for machine vision experts, to move towards automating the setup planning process by means of sensorrealistic simulations.

show abstract

Modeling realistic optical aberrations to reuse existing drive scene recordings for autonomous driving validation

Cited by 4 publications

References 8 publications

Original image noise reconstruction for spatially-varying filtered driving scenes

Original image noise reconstruction for spatially-varying filtered driving scenes

Spatial precision and recall indices to assess the performance of instance segmentation algorithms

Sensor-Realistic Simulations for Evaluation and Planning of Optical Measurement Systems With an Application to Laser Triangulation

Contact Info

Product

Resources

About