A Compact High-Quality Image Demosaicking Neural Network for Edge-Computing Devices

Abstract

Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.

Make the Most Out of Your Net: Alternating Between Canonical and Hard Datasets for Improved Image Demosaicing

“…Initially, the method identifies difficult patch samples in the original training dataset and divides them into sub-categories.Then, the model is trained in an approach we call cyclic training, using the sub-categories to guide the learning process, in a non-standard scheme, alternating between the entire dataset and the generated sub-categories. In addition, there is a recent trend of developing low-capacity models (less than 50k parameters) for edge devices to perform image demosaicing [19,25,34]. We show that our method is able to effectively utilize the model's capacity and surpass recent relevant works across all benchmarks using fewer number of parameters, showcasing a more efficient solution for low-capacity devices.…”

Bio-Inspired Simple Neural Network for Low-Light Image Restoration: A Minimalist Approach

“…Low-light image processing has become a crucial area of research in recent years, as it plays a significant role in various applications such as surveillance, autonomous vehicles, nighttime photography, and even astronomical imaging. Addressing the challenges posed by low-light conditions, such as reduced visibility, increased noise, and loss of details, is essential for enhancing the performance of computer vision tasks like classification, detection, and action recognition Wang et al 2020a;Ma et al 2021). Deep learning techniques, particularly convolutional neural networks (CNNs), have demonstrated remarkable success in addressing low-light image processing challenges Li et al 2019).…”

“…The work in (Hassan et al 2022) presented a multi-task learning framework that jointly addresses the dehazing and denoising problems, resulting in improved image quality. Another study (Zhang et al 2021) introduced a deep reinforcement learning-based approach for adaptive low-light image enhancement, which optimizes the enhancement parameters to achieve visually pleasing results.…”