Multi-Focus Image Fusion Using U-Shaped Networks With a Hybrid Objective

Li, Huaguang; Nie, Rencan; Cao, Jinde; Guo, Xiaopeng; Zhou, Dongming; He, Kangjian

doi:10.1109/jsen.2019.2928818

Cited by 44 publications

(17 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, we can only compare the performance of our method with the available state-of-the-art supervised methods. These include the Non-Subsampled Contournet Transform (NSCT) [ 29 ], Guided Filtering (GF) [ 35 ], Dense SIFT (DSIFT) [ 33 ], as well as the methods based on Boundary Finding (BF) [ 57 ], Convolutional Neural Network (CNN) [ 18 ], the U-net [ 41 ]; deep unsupervised algorithms FusionDN [ 43 ], MFF-GAN [ 44 ] and U2Fusion [ 42 ]. We implemented these algorithms using code acquired from their respective authors.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Gilani

Qin

Mian

2020

Sensors

View full text Add to dashboard Cite

Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as ‘focused’ or ‘defocused’, and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Most recently, with the U-net being successfully applied in image-to image translation [ 39 ] and pixel-wise regression [ 40 ]; a U-net based end-to-end multifocus image algorithm was introduced in [ 41 ]. This method also needs the ground truth for training the U-net fusion network model.…”

Section: Related Workmentioning

confidence: 99%

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Gilani

Qin

Mian

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Multi-focus image fusion has also been achieved through deep neural networks in several ways; the majority of these techniques rely on the detection of the focused region [ 50 ]. In the fusion method presented in [ 50 ], features are extracted through a u-shaped network to obtain high- and low-level frequency texture information. It directly maps multi-focus images to fused images instead of detecting focused regions.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

Exploiting Superpixels for Multi-Focus Image Fusion

Ilyas

Farid

Khan

et al. 2021

Entropy

View full text Add to dashboard Cite

Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. This makes it useful for numerous applications in image enhancement, remote sensing, object recognition, medical imaging, etc. This paper presents a novel multi-focus image fusion algorithm that proposes to group the local connected pixels with similar colors and patterns, usually referred to as superpixels, and use them to separate the focused and de-focused regions of an image. We note that these superpixels are more expressive than individual pixels, and they carry more distinctive statistical properties when compared with other superpixels. The statistical properties of superpixels are analyzed to categorize the pixels as focused or de-focused and to estimate a focus map. A spatial consistency constraint is ensured on the initial focus map to obtain a refined map, which is used in the fusion rule to obtain a single all-in-focus image. Qualitative and quantitative evaluations are performed to assess the performance of the proposed method on a benchmark multi-focus image fusion dataset. The results show that our method produces better quality fused images than existing image fusion techniques.

show abstract

“…The size of each image is 1936×1216 pixels, and some example images are shown in Figure 9. In the following experiments on the multi-focus microscopic image dataset of cancer cells, the proposed method is compared with other well-known multi-focus image fusion methods, namely the LP [1], non-subsampled contourlet transform (NSCT) [6], DWT [3], DTCWT [7], curvelet transform (CVT) [5], CNN-shaped transform [24], and U-shaped transform [27]. In addition, the parameters of these methods are set to the recommended values from their original papers.…”

Section: Experimental Setmentioning

confidence: 99%

“…that are not visible in an end-to-end manner. Li et al [27] used the U-NET to perform semantic segmentation on two non-fully focused images to be fused to form a decision graph to guide fusion. Compared with the CNN-based methods, this method can better distinguish the focus and defocus areas in the source images and reduce the time complexity.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Multi-Focus Biomedical Microscopic Image Fusion Based on m-SegNet

Pei

Yao

et al. 2021

IEEE Photonics J.

View full text Add to dashboard Cite

Activity level measurement and fusion rules are the two key factors of image fusion. In the fusion method based on neural networks, the activity level measurements are realized by dividing the image into small blocks and predicting the sharpness of each block; then, the global decision graph guiding fusion is generated according to the predicted results. However, these two tasks are serial in nature, which makes it difficult to complete them simultaneously while achieving satisfactory fusion performance. Therefore, a new multi-focus microscopic image fusion method is proposed in this paper to quickly fuse multiple histological microscopic images from different focusing planes to generate full-focus images. The improved SegNet network was used to detect the unfocused regions. Considering that two or more images are needed for fusion, a parallel fusion strategy is proposed herein to generate clear fusion images based on multiple images instead of pairwise decision graphs. Compared with the convolutional neural network, the proposed network has better representation ability and can extract and fuse the most ideal features to provide a more accurate fusion decision. Compared with the traditional Segnet network, it is lightweight, which greatly improves computing speed and achieves real-time fusion.

show abstract

Multi-Focus Image Fusion Using U-Shaped Networks With a Hybrid Objective

Cited by 44 publications

References 41 publications

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Exploiting Superpixels for Multi-Focus Image Fusion

Real-Time Multi-Focus Biomedical Microscopic Image Fusion Based on m-SegNet

Contact Info

Product

Resources

About