SESF-Fuse: an unsupervised deep model for multi-focus image fusion

Ma, Boyuan; Zhu, Yu; Yin, Xiang; Ban, Xiaojuan; Huang, Haiyou; Mukeshimana, Michele

doi:10.1007/s00521-020-05358-9

Cited by 169 publications

(85 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiments, we compare the proposed IM-Net with several SOTAs, including DCTVar [8], DSIFT [10], CNN [12], MFF-GAN [18] and SESF [17]. Among them, DCTVar and DSIFT are traditional methods, which are based on transform domain and spatial domain respectively.…”

Section: Methods For Comparisonmentioning

confidence: 99%

“…Specifically, Prabhakar et al [15] took a pre-trained auto-encoder to extract features and mapped the original image into a highdimensional feature space. Since it is found that the extracted deep features have good generalization, robustness, and development potential, some improvements have been made based on this work [16,17]. Recently, an unsupervised fusion method called MFF-GAN [18], played an adversarial game under a joint gradient constraint.…”

Section: Introductionmentioning

confidence: 99%

“…(ii) Inspired by DIP [1], two generative networks are applied to model the deep priors of the clean fused image and the focus map simultaneously, combining the advantages of the method based on focus map estimation that can well preserve the information of source images and the method based on fused image generation that can provide good visual results. (iii) Several SOTAs [8,10,12,18,17] are compared to our approach to demonstrate the effectiveness of our IM-Net.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Zero-Shot Multi-Focus Image Fusion

Jiang¹,

Liu²,

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Multi-focus image fusion (MFIF) is an effective way to eliminate the out-of-focus blur generated in the imaging process. The difficulties in focus level estimation and the lack of real training set for supervised learning make MFIF remain a challenging task after decades of research. According to DIP [1], a neural network can capture the low-level statistics of a single image and can be used as a prior for solving many low-level problems. Based on this idea, we propose a novel architecture named IM-Net comprised of I-Net to model the deep prior of the fused image and M-Net to model the deep prior of the focus map. Without any large scale training set, our method achieves zero-shot learning through the extracted prior information. Experiments on extensively used dataset demonstrate the effectiveness of our approach.

show abstract

Section: Methods For Comparisonmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Shot Multi-Focus Image Fusion

Jiang¹,

Liu²,

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…Existing image fusion methods can be classified into two categories: traditional methods [1,3,21,22,23,24,25,26,27,28,29,30,31] and deep learningbased methods [17,18,19,20,32,33,34,35,36,37]. Although traditional image fusion methods have achieved promising performance before the deep learning era, the hand-crafted feature extraction approaches limit further performance improvement.…”

Section: Introductionmentioning

confidence: 99%

TransFuse: A Unified Transformer-based Image Fusion Framework using Self-supervised Learning

Qu¹,

Liu²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image fusion is a technique to integrate information from multiple source images with complementary information to improve the richness of a single image. Due to insufficient task-specific training data and corresponding ground truth, most existing end-to-end image fusion methods easily fall into overfitting or tedious parameter optimization processes. Two-stage methods avoid the need of large amount of task-specific training data by training encoder-decoder network on large natural image datasets and utilizing the extracted features for fusion, but the domain gap between natural images and different fusion tasks results in limited performance. In this study, we design a novel encoder-decoder based image fusion framework and propose a destruction-reconstruction based selfsupervised training scheme to encourage the network to learn task-specific features. Specifically, we propose three destruction-reconstruction self-supervised auxiliary tasks for multi-modal image fusion, multi-exposure image fusion and multi-focus image fusion based on pixel intensity non-linear transformation, brightness transformation and noise transformation, respectively. In order to encourage different fusion tasks to promote each other and increase the generalizability of the trained network, we integrate the three self-supervised auxiliary

show abstract

“…FIGURE 5 Qualitative comparison of CFNet with different deep learning-based methods. From left to right and top to bottom: Source A, Source B, CNN[6], SESF-Fuse[12], IFCNN[44], DRPL[13], CFNet and the corresponding difference images obtained by subtracting source image B from each fused image, respectively…”

mentioning

confidence: 99%

CFNet: Context fusion network for multi‐focus images

Zhang

Yuan

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Multi-focus image fusion aims to generate a clear image by fusing multiple source images. Existing deep learning-based fusion methods often neglect the context information resulting in the loss of detail information. To address this issue, a context fusion network to merge multi-focus images, namely CFNet, is proposed. Specifically, a context fusion module is proposed to make full use of low-level pixels and high-level semantic features. Particularly, the pyramid fusion mechanism and cross-scale transfer strategy are adopted to ensure the visual and semantic consistency of the fused image. Meanwhile, to extract salient features more effectively, a spatial attention mechanism is introduced to enhance these features. Further, the pyramid loss is used to progressively refine the fused features at each scale. Experimental results show that the proposed method is superior to some existing methods in both qualitative and quantitative evaluation.

show abstract

SESF-Fuse: an unsupervised deep model for multi-focus image fusion

Cited by 169 publications

References 43 publications

Zero-Shot Multi-Focus Image Fusion

Zero-Shot Multi-Focus Image Fusion

TransFuse: A Unified Transformer-based Image Fusion Framework using Self-supervised Learning

CFNet: Context fusion network for multi‐focus images

Contact Info

Product

Resources

About