MLFFusion: Multi-level feature fusion network with region illumination retention for infrared and visible image fusion

Wang, Chuanyun; Sun, Dongdong; Gao, Qian; Wang, Linlin; Yan, Zhuo; Wang, Jingjing; Wang, Ershen; Wang, Tian

doi:10.1016/j.infrared.2023.104916

Cited by 4 publications

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the robustness of the IR camera to illumination and weather changes, we try to additionally introduce thermal infrared (IR) spectroscopy. IR images measure the temperature of the detected target, thus avoiding the effects of low illumination scenarios and foggy weather scenarios on target detection accuracy [21,22]. When the target is in a scene with insufficient visible light, the target features in the RGB image cannot be extracted, and the corresponding IR image can provide effective information about the object.…”

Section: Introductionmentioning

confidence: 99%

Object Detection in Multispectral Remote Sensing Images Based on Cross-Modal Cross-Attention

Zhao,

Ye,

2024

Sensors

View full text Add to dashboard Cite

In complex environments a single visible image is not good enough to perceive the environment, this paper proposes a novel dual-stream real-time detector designed for target detection in extreme environments such as nighttime and fog, which is able to efficiently utilise both visible and infrared images to achieve Fast All-Weatherenvironment sensing (FAWDet). Firstly, in order to allow the network to process information from different modalities simultaneously, this paper expands the state-of-the-art end-to-end detector YOLOv8, the backbone is expanded in parallel as a dual stream. Then, for purpose of avoid information loss in the process of network deepening, a cross-modal feature enhancement module is designed in this study, which enhances each modal feature by cross-modal attention mechanisms, thus effectively avoiding information loss and improving the detection capability of small targets. In addition, for the significant differences between modal features, this paper proposes a three-stage fusion strategy to optimise the feature integration through the fusion of spatial, channel and overall dimensions. It is worth mentioning that the cross-modal feature fusion module adopts an end-to-end training approach. Extensive experiments on two datasets validate that the proposed method achieves state-of-the-art performance in detecting small targets. The cross-modal real-time detector in this study not only demonstrates excellent stability and robust detection performance, but also provides a new solution for target detection techniques in extreme environments.

show abstract

Section: Introductionmentioning

confidence: 99%