Infrared and visible video fusion method based on local differential correlation features and PID control

Abstract

The purpose of infrared and visible video fusion is to combine the complementary features of videos from different modalities. Most fusion algorithms ignore the feature associations of adjacent frames and the guidance of the source frames to the fusion process. Therefore, this paper proposes a new fusion method. First, a spatial-domain multi-attribute detail feature extraction model is proposed, which is capable of extracting the base layer, the bright detail layer and the dark detail layer of the source frames separately. Then, we propose an associated feature extraction model for adjacent frames, which improves the spatial continuity of the fused video. Furthermore, an exponential homomorphic filter is proposed, which simultaneously increases the dynamic range and contrast of the source infrared frames to obtain a general salient target extraction model. In the feature fusion stage, a weighted fusion rule based on edge intensity is used in the detail layer. Then we design the controller, transfer function and measurement function separately, so as to construct a closed-loop proportional-integral-derivative (PID) control system to fuse the base layer, which ensures that the fused video maintains more information of the source video. Experiments on public datasets demonstrate that our fusion method outperforms some state-of-the-art algorithms.