Deep bilateral learning for real-time image enhancement

Abstract

12 megapixel 16-bit linear input (tone-mapped for visualization) tone-mapped with HDR+ 400 -600 ms processed with our algorithm 61 ms, PSNR = 28.4 dB

Multitask bilateral learning for real‐time image enhancement

“…Recently, an approach combining joint upsampling and the DNN has been proposed for real‐time image processing . Deep guided filtering applies the guided filter to the deep learning structure.…”

“…Deep bilateral learning applies bilateral guided upsampling to a deep learning structure. It estimates a 3D bilateral grid by analyzing a low‐resolution input image through DNN and also transforms the full‐resolution input image to a gray‐scale image to create a guidance map.…”

“…This approach has the advantage in that it can increase speed but has a limitation in that the overall processing time is determined by the type of image processing operator. In addition, it is difficult to implement an implicitly defined operation, which can be obtained from a database of human‐annotated input/output pairs …”

“…However, because DNNs usually require a large number of parameters and high computational cost for high performance, these methods are not suitable for real‐time applications. In recent years, algorithms combined with both joint upsampling and deep learning approaches have been actively researched . These algorithms process images by DNN in low resolution and restore them to full resolution through a joint upsampling operator.…”

“…First, the proposed algorithm can reduce both computational cost and the number of parameters because it exploits DNN in low resolution. After DNN processing, the proposed algorithm uses a bilateral guided upsampling operation to restore an image to full resolution. Second, the proposed algorithm has high performance in local image processing as well as global image processing, because the proposed algorithm can efficiently expand the receptive field through atrous convolution and a pyramid pooling module .…”