An investigation of image fusion algorithms using a visual performance-based image evaluation methodology

Abstract

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. SPONSOR/MONITOR'S ACRONYM(S) HPW/RHCV SPONSORING/MONITORING AGENCY REPORT NUMBER AFRL-RH-WP-TR-2009-0015 DISTRIBUTION AVAILABILITY STATEMENTApproved for public release; distribution unlimited. SUPPLEMENTARY NOTES88 ABW PA Cleared 02/17/09; 88ABW-09-0522. ABSTRACTIt is believed that the fusion of multiple different images into a single image should be of great benefit to warfighters engaged in a search task. As such, more research has focused on the improvement of algorithms designed for image fusion. Many different fusion algorithms have already been developed; however, the majority of these algorithms have not been assessed in terms of their visual performance-enhancing effects using militarily relevant scenarios. The goal of this research is to apply a visual performance-based assessment methodology to assess four algorithms that are specifically designed for fusion of multispectral digital images. The image fusion algorithms used included a Principle Component Analysis based algorithm, a Shift-invariant Wavelet transform algorithm, a Contrast-based algorithm, and pixel averaging. The methodology used has been developed to acquire objective human visual performance data as a means of evaluating the image fusion algorithms. Standard objective performance metrics (response time and error rate), were used to compare the fused images vs. two baseline conditions comprising each individual image used in the fused test images. Observers searched images for a military target hidden among foliage and then indicated in which quadrant of the screen the target was located using a spatial-forced-choice paradigm. Response time and percent correct were measured for each observer.

Object recognition methodology for the assessment of multi-spectral fusion algorithms: phase 1

“…In order to investigate the efficacy of different fusion algorithms, [1] the approach was to record the raw imagery from each of three sensors at 25 frames per second. These sequences of images could then be played back in an AVI file, producing a dynamic image for subjects to observe.…”

“…The fusion of multi-spectral images and video sources is emerging as a vital technique for surveillance purposes, object detection, vehicle navigation, and object tracking applications [1] . Purportedly, the main goal of image fusion is to provide a single compact representation of the input images that is more informative than each of the individual inputs.…”

The perceptual processing of fused multi-spectral imagery

“…Despite being a relatively limited literature, human performance with fused imagery has been used with a range of basic visual tasks including detection (Krebs et al, 1999), discrimination (e.g., whether a global scene is upright or vertically inverted; Krebs & Sinai, 2002, Toet et al, 1997), recognition (Ryan & Tinkler, 1995; Sinai, McCarley, & Krebs, 1999; Toet & Franken, 2003), and visual search (Neriani, Pinkus, & Dommett, 2008). This research has been conducted in contexts including aviation (Ryan & Tinkler, 1995; Steele & Perconti, 1997) and surveillance (Neriani et al, 2008; Toet & Franken, 2003; Toet et al, 1997).…”

“…This research has been conducted in contexts including aviation (Ryan & Tinkler, 1995; Steele & Perconti, 1997) and surveillance (Neriani et al, 2008; Toet & Franken, 2003; Toet et al, 1997). Among these applications, there is a wide range of reported results and overall conclusions.…”

Assessment method to fusion effect based on structural similarity comparison in fusion images

“…An objective fusion metric can measure the ability of the fusion process to transfer as accurately as possible this information into the output image. Wang applied structural similarity metric (SSIM) to study JPEG and JPEG2000 compression in 2002 [2] . Successfully SSIM has been used for signal fidelity/quality assessment in many applications, including palm print verification, visual surveillance, image/video compression, watermarking, color image quality and image fusion, etc.…”