Single-Image Visibility Restoration: A Machine Learning Approach and Its 4K-Capable Hardware Accelerator

Abstract

In recent years, machine vision algorithms have played an influential role as core technologies in several practical applications, such as surveillance, autonomous driving, and object recognition/localization. However, as almost all such algorithms are applicable to clear weather conditions, their performance is severely affected by any atmospheric turbidity. Several image visibility restoration algorithms have been proposed to address this issue, and they have proven to be a highly efficient solution. This paper proposes a novel method to recover clear images from degraded ones. To this end, the proposed algorithm uses a supervised machine learning-based technique to estimate the pixel-wise extinction coefficients of the transmission medium and a novel compensation scheme to rectify the post-dehazing false enlargement of white objects. Also, a corresponding hardware accelerator implemented on a Field Programmable Gate Array chip is in order for facilitating real-time processing, a critical requirement of practical camera-based systems. Experimental results on both synthetic and real image datasets verified the proposed method’s superiority over existing benchmark approaches. Furthermore, the hardware synthesis results revealed that the accelerator exhibits a processing rate of nearly 271.67 Mpixel/s, enabling it to process 4K videos at 30.7 frames per second in real time.

Visibility Restoration: A Systematic Review and Meta-Analysis

“…Although the CAP was a fast and straightforward solution, the results were affected by color distortion, background noise, and post-dehazing false enlargement of white objects. These limitations were addressed by Ngo et al [53,54] through adaptive weighting, low-pass filtering, and atmospheric light compensation, respectively. Nevertheless, CAP-based dehazing algorithms appeared to be ineffective against dense haze scenes.…”

“…Similar to the previous subsection, a branching diagram shown in Figure 10 provides a quick overview of the aforementioned introduction of dehazing algorithms utilizing machine learning techniques. Random forest regression [54] Least squares regression [55][56][57][58] Adaptive regularization [60] Total variation regularization [61,62,64,65] L2 regularization [63] Sparsity regularization [65] Information loss [67] Bayesian framework [69] Inhomogeneous Laplacian-Markov random field [70] Local consistent Markov random field [71] Nelder-Mead direct search [73] Huber loss exploitation [74] Fibonacci search [72] Independent component analysis [75] Dictionary learning [76] Radial basis function [77] k-means clustering [79][80][81][82] Semantic-guided regularization [68] Figure 10. Branching diagram summarizing machine-learning-based dehazing algorithms.…”

“…An image processing algorithm would be a workable solution to a particular problem if it could meet the real-time processing requirements, which is, the capability to handle at least 25 fps and fit into an end-device with limited computing resources. Similar to our previous research [54], six dehazing methods corresponding to three main categories were selected to evaluate the processing time. This experiment was conducted on a computer with an Intel Core i9-9900K (3.6 GHz) CPU, 64 GB RAM, and NVIDIA TITAN RTX GPU.…”

Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing

“…For instance, in Reference [ 38 ], the authors implemented a weighted median filtering mechanism to filter out impulsive noise with the least area overhead. In Reference [ 39 ], the authors present an efficient modified hybrid median filter-based implementation that can improve the visibility of high-quality incoming images with a high frame rate. For this work, a similar technique can be adapted to generate inference-free data while keeping the hardware cost minimal.…”

CNN2Gate: An Implementation of Convolutional Neural Networks Inference on FPGAs with Automated Design Space Exploration

“…In [28], the authors proposed a hardware implementation of a haze removal method exploiting adaptive filtering. Additionally, in [29], the authors provided an FPGA implementation of a novel method to recover clear images from degraded ones on FPGA.…”