Image Similarity Metrics Suitable for Infrared Video Stabilization during Active Wildfire Monitoring: A Comparative Analysis

Abstract

Aerial Thermal Infrared (TIR) imagery has demonstrated tremendous potential to monitor active forest fires and acquire detailed information about fire behavior. However, aerial video is usually unstable and requires inter-frame registration before further processing. Measurement of image misalignment is an essential operation for video stabilization. Misalignment can usually be estimated through image similarity, although image similarity metrics are also sensitive to other factors such as changes in the scene and lighting conditions. Therefore, this article presents a thorough analysis of image similarity measurement techniques useful for inter-frame registration in wildfire thermal video. Image similarity metrics most commonly and successfully employed in other fields were surveyed, adapted, benchmarked and compared. We investigated their response to different camera movement components as well as recording frequency and natural variations in fire, background and ambient conditions. The study was conducted in real video from six fire experimental scenarios, ranging from laboratory tests to large-scale controlled burns. Both Global and Local Sensitivity Analyses (GSA and LSA, respectively) were performed using state-of-the-art techniques. Based on the obtained results, two different similarity metrics are proposed to satisfy two different needs. A normalized version of Mutual Information is recommended as cost function during registration, whereas 2D correlation performed the best as quality control metric after registration. These results provide a sound basis for image alignment measurement and open the door to further developments in image registration, motion estimation and video stabilization for aerial monitoring of active wildland fires.

Thermal Infrared Video Stabilization for Aerial Monitoring of Active Wildfires

“…Image similarity can be estimated using various metrics, and how image similarity is measured has implications for registration algorithms. After a comparative analysis of various popular alternatives, our previous results encouraged the use of Mutual Information (MI) as similarity metric for TIR image registration in wildfire contexts [45].…”

“…On the other, by computing similarity between the output registered frame and the original -target-frame. Image similarity was measured using 2D correlation as recommended in [45]. Each registered frame was evaluated using as reference the stable version of the same frame, both under idealised and realistic working conditions.…”

“…Our previous work [45] recommended the use of 2D correlation to measure similarity between fire TIR images. Here, we found that 2D correlation can also be useful to estimate overall video stability if used to compare pairs of consecutive frames.…”

Multimodal and Multiview Wound Monitoring with Mobile Devices

“…Then, an optimization is performed by adjusting the thermal camera pose in order to maximize the similarity in temperatures between the synthetic thermal image (projected reference surface thermography) and the real thermal image. Since both synthetic and real thermal images are in the same modality, the similarity indicator chosen to perform this optimization is the intensity-based 2D correlation coefficient [25,26]. Moreover, the 2D correlation coefficient provides an indicator that is easy to interpret: the closer it is to one, the more similar the images are.…”

Computationally Efficient Wildfire Detection Method Using a Deep Convolutional Network Pruned via Fourier Analysis

“…Video-based forest fire detection can be used to replace traditional point-sensor type detectors because a single pan-tilt-zoom type camera can monitor a wide area, detect forest fire and smoke immediately after the start of the wildfire-as long as the smoke is within the viewing range of the camera. Nowadays, with the development of 5G communication [21,22], unmanned aerial vehicles (UAVs) have also become a good option for wildfire surveillance tasks because of their flexibility compared to fixed surveillance towers. However, all of the traditional video-based methods [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] rely on choosing features manually.…”