Original image noise reconstruction for spatially-varying filtered driving scenes

Abstract

Test drives for the development of camera-based automotive algorithms like object detection or instance segmentation are very expensive and time-consuming. Therefore, the re-use of existing databases like COCO or Berkeley Deep Drive by intentionally varying the image quality in a post-processing step promises to save time and money, while giving access to novel image quality properties. One possible variation we investigate is the sharpness of the camera system, by applying spatially varying optical blur models as low-pass filters on the image data. Any such operation significantly changes the amount and distribution of noise, a central property of image quality, which in this context is an undesired side-effect. In this article, a novel method is presented to reconstruct the original camera sensor noise for the filtered image. This is different from denoising. The method estimates the original camera sensor noise using the combination of principal component analysis (PCA) and a variance-stabilizing transformation. The noise is then reconstructed for the filtered image with the PCA applied locally on small image sections, and an inverse variance-stabilizing transformation. Although the resulting noise distribution can slightly deviate from the original, this novel method does not introduce any image artifacts as denoising would do. We present the method as applied to synthetic and real driving scenes at different noise levels and discuss the accuracy of the reconstruction visually and with statistical parameters.