Scanpath Analysis of Fused Multi-Sensor Images with Luminance Change: A Pilot Study

Abstract

-Image fusion is the process of combining images of differing modalities, such as visible and infrared (IR) images. Significant work has recently been carried out comparing methods of fused image assessment, with findings strongly suggesting that a task-centred approach would be beneficial to the assessment process. The current paper reports a pilot study analysing eye movements of participants involved in four tasks. The first and second tasks involved tracking a human figure wearing camouflage clothing walking through thick undergrowth at light and dark luminance levels, whilst the third and fourth task required tracking an individual in a crowd, again at two luminance levels. Participants were shown the original visible and IR images individually, pixelaveraged, contrast pyramid, and dual-tree complex wavelet fused video sequences. They viewed each display and sequence three times to compare intersubject scanpath variability. This paper describes the initial analysis of the eye-tracking data gathered from the pilot study. These were also compared with computational metric assessment of the image sequences.

Scanpath assessment of visible and infrared side-by-side and fused video displays

“…Previous work involving the novel use of an eye-tracking paradigm to analyse participants' scanpaths, compared visible (Viz) and infrared (IR) individual inputs with averaged (AVE), discrete wavelet transform (DWT), and dual-tree complex wavelet transform (DT-CWT) fused displays [1,2]. The present study extends this work by introducing a side-by-side (SBS) or adjacent display comprising both the Viz and IR inputs shown simultaneously.…”

“…Different fusion methods have been shown to lead to increased and longer fixations in a target detection task, with a principal components fusion scheme leading to greatest number of fixations (whether this necessarily a positive feature of the fusion scheme is not fully explored) [20]. Other work [1,2] has shown that different fusion methods can be shown to aid participants to track better figures through video sequences, with the simple AVE fusion being particularly useful in this respect.…”

“…There are several benefits to multi-sensor image fusion including wider spatial and temporal coverage, extended range of operation, decreased uncertainty, improved reliability and increased robustness of system performance. Whilst much academic research has recently focused on the methods for fusing static images, and for the assessment of such images, little work has been carried out with regard to multi-sensor video fusion [1,2,7].…”

The Effect of Pixel-Level Fusion on Object Tracking in Multi-Sensor Surveillance Video

“…The videos described in Section 4.1 were fused using the following methods: Averaging Technique (AVE), Laplacian Pyramid (LP), Discrete Wavelet Transform (DWT) and Dual-Tree Complex Wavelet Transform (DT-CWT) (please see [6] for more details on these techniques). In the multiresolution methods (LP, DWT and DT-CWT) a 5-level decomposition is used and fusion is performed by selecting the coefficient with the maximum absolute value, except for the case of the lowest resolution subband where the mean value is used.…”

Cognitive Image Fusion and Assessment

“…However, until now the human end user has not been involved in the design process and the development of image fusion algorithms to any great extent. Mostly, image fusion algorithms are developed in isolation, and the human end-user is little more than an afterthought, so that separate follow-up evaluation studies are usually required to assess to what extent humans benefit from these methods (Aguilar et al, 1999;Dixon et al, 2005;Dixon et al, 2006a;Dixon et al, 2006b;Essock et al, 1999;Essock et al, 2005;Krebs & Sinai, 2002;Smith et al, 2002;Toet & Franken, 2003;Waxman et al, 2006). Recently has it been realized that the only way to guarantee the ultimate effectiveness of image fusion methods for human observers is to include human evaluation as an integral part of the design process (Muller & Narayanan, 2009).…”