Perceptual evaluation of colorized nighttime imagery

Abstract

We recently presented a color transform that produces fused nighttime imagery with a realistic color appearance (Hogervorst & Toet, 2010, Information Fusion, 11-2, 69-77). To assess the practical value of this transform we performed two experiments in which we compared human scene recognition for monochrome intensified (II) and longwave infrared (IR) imagery, and color daylight (REF) and fused multispectral (CF) imagery. First we investigated the amount of detail observers can perceive in a short time span (the gist of the scene). Participants watched brief image presentations and provided a full report of what they had seen. Our results show that REF and CF imagery yielded the highest precision and recall measures, while both II and IR imagery yielded significantly lower values. This suggests that observers have more difficulty extracting information from monochrome than from color imagery. Next, we measured eye fixations of participants who freely explored the images. Although the overall fixation behavior was similar across image modalities, the order in which certain details were fixated varied. Persons and vehicles were typically fixated first in REF, CF and IR imagery, while they were fixated later in II imagery. In some cases, color remapping II imagery and fusion with IR imagery restored the fixation order of these image details. We conclude that color remapping can yield enhanced scene perception compared to conventional monochrome nighttime imagery, and may be deployed to tune multispectral image representation such that the resulting fixation behavior resembles the fixation behavior for daylight color imagery.

Colorization of Multispectral Image Fusion using Convolutional Neural Network approach

“…Xingchen Zhang and et al [28] represented the visible and infrared image fusion benchmark (VIFB) which is a collection of image pairs (21), code library with various fusion algorithms (20), and assessment parameters (13) that outlines current developments in fusion of visible and infrared images. Authors also run extensive tests to understand the performance of different methods.…”

“…Toet and et al [21] compared the ability of humans to identify scenes using monochrome intensified (II) and long wave infrared images, as well as color daylight and fused multispectral images. As a result of their research, they found the highest accuracy and remembrance metrics, while II and IR photography produced significantly lower metrics.…”