Abstract:As an important branch in the field of image fusion, the multi-focus image fusion technique can effectively solve the problem of optical lens depth of field, making two or more partially focused images fuse into a fully focused image. In this paper, the methods based on boundary segmentation was put forward as a group of image fusion method. Thus, a novel classification method of image fusion algorithms is proposed: transform domain methods, boundary segmentation methods, deep learning methods, and combination… Show more
“…Traditional MFIF methods are typically categorized into two main groups: transform domain methods and spatial domain methods [33]. Transform domain methods mainly operate the decomposition coefficient after image transformation, encompassing three key fusion stages: image transformation, coefficient after image transformation, and inverse transformation reconstruction [34]. According to the application of the image transform, transform domain methods can be further classified into multi-scale decomposition (MSD)-based methods (e.g., Laplacian pyramid [14,35], discrete wavelet transform [36,37], nonsubsampled contourlet transform [38][39][40], neighbor distance filtering [41,42], empirical mode decomposition [43]), sparse representation (SR)-based methods (orthogonal matching pursuit [44,45]), gradient domain (GD)-based methods (structure tensor [46,47]), methods based on other transform (independent component analysis [48], cartoon-texture decomposition [49]) and methods based on the combination of different transforms (curvelet transform and wallet transform [50]).…”
In this study, we introduce the U-Swin fusion model, an effective and efficient transformer-based architecture designed for the fusion of multi-focus microscope images. We utilized the Swin-Transformer with shifted window and path merging as the encoder for extracted hierarchical context features. Additionally, a Swin-Transformer-based decoder with patch expansion was designed to perform the un-sampling operation, generating the fully focused image. To enhance the performance of the feature decoder, the skip connections were applied to concatenate the hierarchical features from the encoder with the decoder up-sample features, like U-net. To facilitate comprehensive model training, we created a substantial dataset of multi-focus images, primarily derived from texture datasets. Our modulators demonstrated superior capability for multi-focus image fusion to achieve comparable or even better fusion images than the existing state-of-the-art image fusion algorithms and demonstrated adequate generalization ability for multi-focus microscope image fusion. Remarkably, for multi-focus microscope image fusion, the pure transformer-based U-Swin fusion model incorporating channel mix fusion rules delivers optimal performance compared with most existing end-to-end fusion models.
“…Traditional MFIF methods are typically categorized into two main groups: transform domain methods and spatial domain methods [33]. Transform domain methods mainly operate the decomposition coefficient after image transformation, encompassing three key fusion stages: image transformation, coefficient after image transformation, and inverse transformation reconstruction [34]. According to the application of the image transform, transform domain methods can be further classified into multi-scale decomposition (MSD)-based methods (e.g., Laplacian pyramid [14,35], discrete wavelet transform [36,37], nonsubsampled contourlet transform [38][39][40], neighbor distance filtering [41,42], empirical mode decomposition [43]), sparse representation (SR)-based methods (orthogonal matching pursuit [44,45]), gradient domain (GD)-based methods (structure tensor [46,47]), methods based on other transform (independent component analysis [48], cartoon-texture decomposition [49]) and methods based on the combination of different transforms (curvelet transform and wallet transform [50]).…”
In this study, we introduce the U-Swin fusion model, an effective and efficient transformer-based architecture designed for the fusion of multi-focus microscope images. We utilized the Swin-Transformer with shifted window and path merging as the encoder for extracted hierarchical context features. Additionally, a Swin-Transformer-based decoder with patch expansion was designed to perform the un-sampling operation, generating the fully focused image. To enhance the performance of the feature decoder, the skip connections were applied to concatenate the hierarchical features from the encoder with the decoder up-sample features, like U-net. To facilitate comprehensive model training, we created a substantial dataset of multi-focus images, primarily derived from texture datasets. Our modulators demonstrated superior capability for multi-focus image fusion to achieve comparable or even better fusion images than the existing state-of-the-art image fusion algorithms and demonstrated adequate generalization ability for multi-focus microscope image fusion. Remarkably, for multi-focus microscope image fusion, the pure transformer-based U-Swin fusion model incorporating channel mix fusion rules delivers optimal performance compared with most existing end-to-end fusion models.
“…Shifted and scaled versions of a wavelet can positively impact the resolution in the time or frequency domain according to the selected parameters. The wavelet transform aims to maintain a good resolution in the time and frequency domain, contrary to the Fourier transform [73,74].…”
Non-destructive testing (NDT) of aerospace structures has gained significant interest, given its non-destructive and economic inspection nature enabling future sustainable aerospace maintenance repair operations (MROs). NDT has been applied to many different domains, and there is a number of such methods having their individual sensor technology characteristics, working principles, pros and cons. Increasingly, NDT approaches have been investigated alongside the use of data fusion with the aim of combining sensing information for improved inspection performance and more informative structural health condition outcomes for the relevant structure. Within this context, image fusion has been a particular focus. This review paper aims to provide a comprehensive survey of the recent progress and development trends in NDT-based image fusion. A particular aspect included in this work is providing critical insights on the reliable inspection of aerospace composites, given the weight-saving potential and superior mechanical properties of composites for use in aerospace structures and support for airworthiness. As the integration of NDT approaches for composite materials is rather limited in the current literature, some examples from non-composite materials are also presented as a means of providing insights into the fusion potential.
“…NDT fusion is a challenging subject given the dissimilar data acquisition environment of uni-modal inspections; there is some work on combining NDT techniques by fusion rules such as generic maximum, minimum or average merging by pixel-wise, principle component analysis (PCA) and wavelets, as well as AI-based approaches [10,[18][19][20][21]. In this study, we use the maximum combination rule, given in equations ( 1), as a simple yet effective synergistic approach to retrieve defect edges.…”
Section: Fig 4 Flow Diagram Of the Proposed Decision-level Fusion App...mentioning
Low energy impact damage in composite materials may be more concerning than it appears visually, often requiring a detailed examination for accurate assessment to ensure safe and sustainable operation. Non-destructive testing (NDT) methods provide such inspection techniques, and in this paper, NDT-based fusion is explored for enhanced identification of defect size and location compared to indepdently using individual NDT methods separately. Three Carbon Fiber Reinforced Polymer (CFRP) specimens are examined, each with an impact damage of a given energy level, using pulsed thermography (PT) and phased array (PA) ultrasonic methods. Following the extraction of binary defect shapes from source images, a decision-level fusion approach is performed. The results indicate that combining ultrasonic and infrared thermography (IRT) inspections for CFRP composite materials is promising to achieve enhanced and improved detection traceability.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.