Two‐stage single image dehazing network using swin‐transformer

Abstract

Hazy images often have color distortion, blur and other visible visual quality degradation, affecting the performance of some advanced visual tasks. Therefore, single image dehazing has always been a challenging and significant problem. Convolutional neural network has been widely used in image dehazing task, but the limitations of convolutional operation limit the development of dehazing task. Nowadays, Transformer offers a holistic approach to CV development and does not grow in location as the network deepens. For this reason, a hierarchical Transformer is introduced for use in the dehazing network. Specifically, the codec is improved and Transformer and CNN are combined to achieve basic feature extraction in the first stage. The encoder only models the global relationship at each layer, reducing the resolution of the feature map continuously and expanding the field of perception. In addition, an inter‐block supervision mechanism is added between encoder unit and decoder unit to refine features and supervise and select them, thus improving the efficiency of feature transmission. In the second stage, the original resolution block is used to extract the local features, and then feature fusion and interaction are carried out. In addition, to ensure the authenticity of the transmission of characteristic signals in the first stage and improve the transmission efficiency of the network, fusion attention mechanism is added between stages. It adds the residual image of the early input features to the image acquired in the first stage, then passes to the next stage. Ablation experiments show that the two‐stage network has significant benefits for image quality and visual effects. The experimental results on RESIDE, O‐Haze, and I‐Haze datasets show that the method is superior to advanced methods in dehazing effectiveness.

Multi‐modal object detection via transformer network

“…The initial design operation is to multiply the two features to obtain a matrix, then sum pool the matrix to obtain the feature vector, and then use this vector to classify, but it usually suffers from as a high computational complexity as $$\mathcal {O}(n^2)$$. In recent years, an effective attention‐based fusion method has been developed by extending transformer [22–27]. The self‐attention mechanism in Transformer can be regarded as information fusion on a fully‐connected graph, which is more general to model the input data.…”