Critical Reflection on Quantitative Assessment of Image Fusion Quality

Abstract

Abstract. Image fusion technique has been extended its development from multi-sensor fusion, multi-model fusion to multi-focus fusion. More and more advanced techniques such as deep learning have been integrated into the development of image fusion algorithms. However, as an important aspect, fusion quality assessment has been received less attention. This paper intends to reflect on the commonly used indices for quantitative assessment and investigate how they can represent the fusion quality regarding spectral preservation and spatial improvement. We found that image dissimilarities are unavoidable due to the spectral coverage of different image sensors. Image fusion should integrate these dissimilarities when they are representing spatial improvement. Such integration will naturally change the pixel values. However, as the quality indices for the assessment of spectral preservation are measuring image dissimilarities, the integration of spatial information will lead to a low fusion quality assessment. For the evaluation of spatial improvement, the quality indices only work if the spatial details have been lost; however, in the case of spatial details gain, these indices do not reflect them as spatial improvements. Moreover, this paper raises attention to image processing procedures involved in image fusion, including image geo-registration, image clipping and image resampling, which will change image statistics and thereby influence the quality assessment when statistical indices are used.