DarkVisionNet: Low-Light Imaging via RGB-NIR Fusion with Deep Inconsistency Prior

Abstract

RGB-NIR fusion is a promising method for low-light imaging. However, high-intensity noise in low-light images amplifies the effect of structure inconsistency between RGB-NIR images, which fails existing algorithms. To handle this, we propose a new RGB-NIR fusion algorithm called Dark Vision Net (DVN) with two technical novelties: Deep Structure and Deep Inconsistency Prior (DIP). The Deep Structure extracts clear structure details in deep multiscale feature space rather than raw input space, which is more robust to noisy inputs. Based on the deep structures from both RGB and NIR domains, we introduce the DIP to leverage the structure inconsistency to guide the fusion of RGB-NIR. Benefits from this, the proposed DVN obtains high-quality low-light images without the visual artifacts. We also propose a new dataset called Dark Vision Dataset (DVD), consisting of aligned RGB-NIR image pairs, as the first public RGB-NIR fusion benchmark. Quantitative and qualitative results on the proposed benchmark show that DVN significantly outperforms other comparison algorithms in PSNR and SSIM, especially in extremely low light conditions.

Enhancement of dark areas on the surface of scrap metals based on RGB-NIR image fusion

“…The structural features are represented by binary edge features, obtained through the application of the Sobel operator for image filtering. The inconsistency between images is defined as follows: 24 F(edgeC,edgeN)=λ(1−edgeC)(1−edgeN)+edgeC·edgeN,where edgeC represents the edge feature map for each color channel of the RGB image and edgeN represents the edge feature map for the NIR image, with the dimensions of the feature map being the same as those of the original images. Figure 4 shows the representation of structural inconsistency between RGB and NIR images.…”