Quality assessment of false-colored fused displays

Abstract

Abstract— The problem of assessing the quality of fused images (composites created from inputs of differing modalities, such as infrared and visible light radiation) is an important and growing area of research. Recent work has shown that the process of assessing fused images should not rely entirely on subjective quality methods, with objective tasks and computational metrics having important contributions to the assessment procedure. The current paper extends previous findings, applying a psychophysical selection task, metric evaluation, and subjective quality judgments to a range of fused surveillance images. Fusion schemes included the contrast pyramid and shift invariant discrete wavelet transform (Experiment 1), the complex wavelet transform (Experiments 1 & 2), and two false‐coloring methods (Experiment 2). In addition, JPEG2000 compression was applied at two levels, as well as an uncompressed control. Reaction time results showed the contrast pyramid to lead to slowest performance in the objective task, whilst the presence of color greatly reduced reaction times. These results differed from both the subjective and metric results. The findings support the view that subjective quality ratings should be used with caution, especially if not accompanied by some task.

Scanpath assessment of visible and infrared side-by-side and fused video displays

“…Initiated by Toet and colleagues [13,14], major advances have been made in applying some form of task to the assessment process, and moving away from the ever-present subjective quality assessment. Furthermore, in recent findings [10][11][12], it has been shown that objective task results can differ significantly from subjective ratings. It is thus essential to choose a well-defined and relevant task when assessing fused images or video sequences, and to go beyond simply applying a subjective rating to the fused outputs.…”

“…This method has greater directional selectivity than the DWT and is shift invariant with reduced over completeness. DT-CWT has been shown to produce better results in terms of image fusion than other wavelet methods [10,11], as well as other pyramid and averaging methods [12], across a range of qualitative and quantitative assessments. These advantages come however at the cost of greater computational expense.…”

Task-based scanpath assessment of multi-sensor video fusion in complex scenarios

“…The DT-CWT has been adapted by Nikolov and colleagues [41] to enable it to be used for image fusion. When used in human assessment, the DT-CWT has been shown to perform better than the DWT in subjective comparison [41], as well as outperforming other fusion methods in a range of objective and subjective assessment tasks [42][43][44].…”

“…In the current context, the use of subjective ratings tasks is not viable, as they are particularly time-consuming and troublesome with video assessment processes. Moreover, it is worth noting that subjective ratings have been shown to lead to differing patterns of results from objective human tasks [42][43][44], making their value in real terms questionable.…”

“…Dixon and colleagues have also recently embarked on a broad range of fused image testing [1][2][3][42][43][44]. Based on the previous findings suggesting an overall benefit of fusion in given scenarios, we have focused on comparing different fusion methods including AVE, a contrast pyramid (CP), DWT, SIDWT and DT-CWT.…”