Multiple Object Detection and Tracking in the Thermal Spectrum

Ahmar, Wassim A. El; Kolhatkar, Dhanvin; Nowruzi, Farzan Erlik; AlGhamdi, Hamzah; Hou, Jonathan; Laganière, Robert

doi:10.1109/cvprw56347.2022.00042

Cited by 6 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FLIRs [37]. This dataset was released in July 2018, with a total of 14, 000 images, 10, 000 of which are from short video clips, and another 4, 000 BONUS images are from a 140-second video.…”

Section: Data Preparationmentioning

confidence: 99%

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection

Meng,

Si,

Mao

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Compared with visible light images, thermal infrared images have poor resolution, low contrast, signal-to-noise ratio, blurred visual effects, and less information. Thermal infrared sports target detection methods relying on traditional convolutional networks capture the rich semantics in high-level features but blur the spatial details. The differences in physical information content and spatial distribution of high and low features are ignored, resulting in a mismatch between the region of interest and the target. To address these issues, we propose a local attention-guided Swin-transformer thermal infrared sports object detection method (LAGSwin) to encode sports objects’ spatial transformation and orientation information. On the one hand, Swin-transformer guided by local attention is adopted to enrich the semantic knowledge of low-level features by embedding local focus from high-level features and generating high-quality anchors while increasing the embedding of contextual information. On the other hand, an active rotation filter is employed to encode orientation information, resulting in orientation-sensitive and invariant features to reduce the inconsistency between classification and localization regression. A bidirectional criss-cross fusion strategy is adopted in the feature fusion stage to enable better interaction and embedding features of different resolutions. At last, the evaluation and verification of multiple open-source sports target datasets prove that the proposed LAGSwin detection framework has good robustness and generalization ability.

show abstract

Section: Data Preparationmentioning

confidence: 99%

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection

Meng,

Si,

Mao

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Huang et al [22] have tested various patterns of object detection frameworks by modifying different components and parameters to find the best possible configurations for specific scenarios, e.g., deployment on mobile devices. El Ahmar, Wassim A., et al [23] proposed a multi-object detection and tracking method on thermal data along with their corresponding visible data. Their experimental result shows that transfer learning is more effective even on different data modalities such as thermal data when used to train a detector on a dataset from the same spectrum as the initial weights.…”

Section: Background/ Literature Reviewmentioning

confidence: 99%

Development, optimization, and deployment of thermal forward vision systems for advance vehicular applications on edge devices

Farooq

Shariff

Khan

et al. 2023

Fifteenth International Conference on Machine Vision (ICMV 2022)

View full text Add to dashboard Cite

In this research work, we have proposed a thermal tiny-YOLO multi-class object detection (TTYMOD) system as a smart forward sensing system that should remain effective in all weather and harsh environmental conditions using an end-to-end YOLO deep learning framework. It provides enhanced safety and improved awareness features for driver assistance. The system is trained on large-scale thermal public datasets as well as newly gathered novel open-sourced dataset comprising of more than 35,000 distinct thermal frames. For optimal training and convergence of YOLO-v5 tiny network variant on thermal data, we have employed different optimizers which include stochastic decent gradient (SGD), Adam, and its variant AdamW which has an improved implementation of weight decay. The performance of thermally tuned tiny architecture is further evaluated on the public as well as locally gathered test data in diversified and challenging weather and environmental conditions. The efficacy of a thermally tuned nano network is quantified using various qualitative metrics which include mean average precision, frames per second rate, and average inference time. Experimental outcomes show that the network achieved the best mAP of 56.4% with an average inference time/ frame of 4 milliseconds. The study further incorporates the optimization of tiny network variant using the TensorFlow Lite quantization tool which is beneficial for the deployment of deep learning architectures on the edge and mobile devices. For this study, we have used a raspberry pi 4 computing board for evaluating the real-time feasibility performance of an optimized version of the thermal object detection network for the automotive sensor suite. The source code, trained and optimized models and complete validation/ testing results are publicly available at https://github.com/MAli-Farooq/Thermal-YOLO-And-Model-Optimization-Using-TensorFlowLite.

show abstract

“…However, the tracking conditions were limited to fixed cameras, and a plain background. In [25], a tracking-by-detection approach was studied in the thermal spectrum. A convolutional neural network, pre-trained with visible images, was transferred to the thermal tracking [26].…”

Section: Introductionmentioning

confidence: 99%

Thermal Image Tracking for Search and Rescue Missions with a Drone

Yeom

2024

Drones

View full text Add to dashboard Cite

Infrared thermal imaging is useful for human body recognition for search and rescue (SAR) missions. This paper discusses thermal object tracking for SAR missions with a drone. The entire process consists of object detection and multiple-target tracking. The You-Only-Look-Once (YOLO) detection model is utilized to detect people in thermal videos. Multiple-target tracking is performed via track initialization, maintenance, and termination. Position measurements in two consecutive frames initialize the track. Tracks are maintained using a Kalman filter. A bounding box gating rule is proposed for the measurement-to-track association. This proposed rule is combined with the statistically nearest neighbor association rule to assign measurements to tracks. The track-to-track association selects the fittest track for a track and fuses them. In the experiments, three videos of three hikers simulating being lost in the mountains were captured using a thermal imaging camera on a drone. Capturing was assumed under difficult conditions; the objects are close or occluded, and the drone flies arbitrarily in horizontal and vertical directions. Robust tracking results were obtained in terms of average total track life and average track purity, whereas the average mean track life was shortened in harsh searching environments.

show abstract

Multiple Object Detection and Tracking in the Thermal Spectrum

Cited by 6 publications

References 19 publications

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection

Development, optimization, and deployment of thermal forward vision systems for advance vehicular applications on edge devices

Thermal Image Tracking for Search and Rescue Missions with a Drone

Contact Info

Product

Resources

About