Fast saliency-aware multi-modality image fusion

Han, Jungong; Pauwels, Eric J.; Zeeuw, Paul de

doi:10.1016/j.neucom.2012.12.015

Cited by 116 publications

(35 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They define the weight maps by two quality measures, say local contrast and color consistency, in the fusion process. Han et al [38] employed the saliency detection to generate the saliency maps for objects or regions. The MRF model was utilized to combine the saliency map and the co-occurrence of hot spots and motion.…”

Section: Related Workmentioning

confidence: 99%

Robust Match Fusion Using Optimization

Qin

Shen

Mao

et al. 2015

IEEE Trans. Cybern.

View full text Add to dashboard Cite

In this paper, we present a novel patch-based match and fusion algorithm by taking account of moving scene in a multiple exposure image sequence using optimization. A uniform iterative approach is developed to match and find the corresponding patches in different exposure images, which are then fused in each iteration. Our approach does not need to align the input multiple exposure images before the fusion process. Considering that the pixel values are affected by various exposure time, we design a new patch-based energy function that will be optimized to improve the matching accuracy. An efficient patch-based exposure fusion approach using the random walker algorithm is developed to preserve the moving objects from the input multiple exposure images. To the best of our knowledge, our algorithm is the first patch-based exposure fusion work to preserve the moving objects of dynamic scenes that does not need the registration process of different exposure images. Experimental results of moving scenes demonstrate that our algorithm achieves visually pleasing fusion results without ghosting artifacts, while the results produced by the state-of-the-art exposure fusion and tone mapping algorithms exhibit different levels of ghosting artifacts.

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Match Fusion Using Optimization

Qin

Shen

Mao

et al. 2015

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…As a result, it is necessary to find a new path to process both visible and thermal image based on their characteristics. Inspired by several multi-modality image fusion approaches [32][33][34], where color and infrared images are integrated for saliency-based image fusion [32,34] and image registration [33], the fusion of the two image modalities (RGB and thermal) offers new insights for the supplementary information they can provide. This has proven to be a success in determining the refined foreground map by the fusion of both visible and thermal binary maps.…”

Section: Introductionmentioning

confidence: 99%

Cognitive Fusion of Thermal and Visible Imagery for Effective Detection and Tracking of Pedestrians in Videos

Yan¹,

Ren²,

Zhao

et al. 2017

Cogn Comput

View full text Add to dashboard Cite

In this paper, we present an efficient framework to cognitively detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, the thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, cognitive fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by a two-stage background subtraction for foreground detection in visible and thermal images. To deal with cases of occlusion or overlap, knowledge-based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. To evaluate the proposed method, a publicly available color-thermal benchmark dataset Object Tracking and Classification in and Beyond the Visible Spectrum is employed here. For our foreground detection evaluation, objective and subjective analysis against several state-of-the-art methods have been done on our manually segmented ground truth. For our object tracking evaluation, comprehensive qualitative experiments have also been done on all video sequences. Promising results have shown that the proposed fusion-based approach can successfully detect and track multiple human objects in most scenes regardless of any light change or occlusion problem.

show abstract

“…Such a detector that mimics the human visual attention mechanism has been served as a foundation for many computer vision applications including object classification (Peng and Shao, 2015), image segmentation (Fouquier et al, 2012), image retrieval (Chen and Cheng, 2009), image fusion (Han et al, 2013), and image thumbnailing (Marchesotti et al, 2009). …”

Section: Introductionmentioning

confidence: 99%