Local Adaptive Illumination-Driven Input-Level Fusion for Infrared and Visible Object Detection

Abstract

Remote sensing object detection based on the combination of infrared and visible images can effectively adapt to the around-the-clock and changeable illumination conditions. However, most of the existing infrared and visible object detection networks need two backbone networks to extract the features of two modalities, respectively. Compared with the single modality detection network, this greatly increases the amount of calculation, which limits its real-time processing on the vehicle and unmanned aerial vehicle (UAV) platforms. Therefore, this paper proposes a local adaptive illumination-driven input-level fusion module (LAIIFusion). The previous methods for illumination perception only focus on the global illumination, ignoring the local differences. In this regard, we design a new illumination perception submodule, and newly define the value of illumination. With more accurate area selection and label design, the module can more effectively perceive the scene illumination condition. In addition, aiming at the problem of incomplete alignment between infrared and visible images, a submodule is designed for the rapid estimation of slight shifts. The experimental results show that the single modality detection algorithm based on LAIIFusion can ensure a large improvement in accuracy with a small loss of speed. On the DroneVehicle dataset, our module combined with YOLOv5L could achieve the best performance.

Vision-Based On-Road Nighttime Vehicle Detection and Tracking Using Improved HOG Features

“…The experimental results are presented in Figure 8 . Figure 9 a indicates that under different road scenes, there will be misdetections when using the template method [ 29 , 33 ], such as color, coordinate, and shape characteristics. Figure 9 b shows that some vehicles with less obvious contours will fail to be detected when using the traditional HOG feature.…”

Improving YOLOv7-Tiny for Infrared and Visible Light Image Object Detection on Drones

“…Anchor Boxes YOLO-anchor (12,16), (19,36), (40,28); (36,75) (30,16), (28,24), (20,39), (40,20), (18,49), (46, 25), (36,32), (22,55), (29,58) This paper proposes an anchor box assignment method that clusters the bounding boxes of five vehicle categories on the DroneVehicle dataset separately, obtaining 15 anchor boxes. Then, according to the aspect ratio, the anchor boxes are grouped into different network layers: the ones with an aspect ratio less than 1 are assigned to the shallow layer, the ones with an aspect ratio close to 1 are assigned to the deep layer, and the ones with an aspect ratio greater than 1 are assigned to the middle layer.…”

“…In addition, some algorithms explore object detection algorithms that fuse infrared and visible light images. D-ViTDet [21] and LAIIFusion [22] use illumination perception modules to perceive the illumination difference in each region of the image, providing more suitable reference for cross-modal image fusion. UA-CMDet [23], RISNet [24], and ECISNet [25] optimize the cross-modal mutual information utilization to improve the detection performance.…”

Res-SwinTransformer with Local Contrast Attention for Infrared Small Target Detection

“…Zhu et al [24] proposed an unsupervised infrared object detection framework based on spatial-temporal patch tensor and object selection. Apart from that, there are also methods that use complementary information from infrared and visible images to detect small targets [25][26][27]. Although the detection speed and accuracy of the deep learning-based method have been improved, the information of small infrared targets is gradually reduced or even lost as the convolutional layers are stacked, which makes it difficult for the detector to extract effective features.…”