Fusion of hyperspectral and multispectral image by dual residual dense networks

Qiu, Kang; Yi, Benshun; Xiang, Mian; Zheng, Xiao

doi:10.1117/1.oe.58.2.023110

Cited by 2 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Residual dense network combines shallow and deep features together, adaptively learning more effective features, allowing the network to make full use of the hierarchical features of the original LR images, and showing significant advantages in image SR task (Wen et al 2018). Inspired by the above-described approaches, Qiu et al (2019) proposed a dual residual dense network (DRDNs), the fusion of hyperspectral image (HSI) and the multispectral image (MSI) were achieved. As shown in figure 2, DRDNs is composed of HSI feature extraction network, MSI feature extraction network, and MSI/HSI feature fusion network.…”

Section: Hyper Densely Connection Networkmentioning

confidence: 99%

HyperTDP-Net: A Hyper-densely Connected Compression-and-Decomposition Network Based on Trident Dilated Perception for PET and MRI Image Fusion

Wang

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Conventional methods based on deep learning for medical image fusion usually only connect source images by convolution operations of separate paths to extract local features, without considering their global features, which often leads to the problem of unclear detail information in the final fusion images. Toward this end, we propose a novel end-to-end fusion model for PET and MRI images to achieve information interaction between different pathways, termed as Hyper-densely connected compression-and-decomposition Network based on Trident Dilated Perception for PET and MRI Image Fusion (HyperTDP-Net). In particular, in the compression network, a dual residual hyper densely module is constructed to take full advantage of middle layer information. Moreover, we establish the trident dilated perception module to precisely determine the location information of features, and improve the feature representation capability of the network. In addition, we abandon the ordinary mean square error as the content loss function and propose a new content-aware loss consisting of structural similarity loss and gradient loss, so that the fused image not only contains rich texture details but also maintains sufficient structural similarity with the source images. Abundant experiments demonstrate that our HyperTDP-Net obtains significant fusion performance, which exceeds other advanced fusion methods in terms of qualitative visual description and quantitative assessment.

show abstract

Section: Hyper Densely Connection Networkmentioning

confidence: 99%