SE‐Swin: An improved Swin‐Transfomer network of self‐ensemble feature extraction framework for image retrieval

Abstract

The Swin‐Transformer is a variant of the Vision Transformer, which constructs a hierarchical Transformer that computes representations with shifted windows and window multi‐head self‐attention. This method can handle the scale invariance problem and performs well in many computer vision tasks. In image retrieval, high‐quality feature descriptors are necessary to improve retrieval accuracy. This paper proposes a self‐ensemble Swin‐Transformer network structure to fuse the features of different layers of the Swin‐Transformer network, eliminating noise points present in a single layer, and improving the retrieval effect. Two experiments were conducted, one on the In‐shop Clothes Retrieval dataset and another on the Stanford Online Product dataset. The experiments showed that the proposed method significantly increased the retrieval effect of features extracted using Vision Transformer, surpassing previous state‐of‐the‐art image retrieval methods. In the second experiment, the feature map of the trained model was visualized, revealing that the improved network significantly reduces focus on some noise points and enhances focus on image features compared to the original network.

Structural damage identification method based on Swin Transformer and continuous wavelet transform

“…Swin Transformer [19] has gained popularity in various fields due to its exceptional image processing capabilities. For example, Üzen et al utilized Swin Transformer to detect surface defects at the pixel level [20] . Xu et al proposed a self-integrated Swin Transformer network structure, which combines the features of different layers of the Swin Transformer network and removes noisy points present in a single layer, thereby enhancing the retrieval performance [21] .…”

A Recognition and Classification Method for Underground Acoustic Emission Signals Based on Improved CELMD and Swin Transformer Neural Networks

“…Dosovitskiy et al applied the Transformer neural network to image classification, proposing the ViT (Vision Transformer) neural network, which broke the monopoly of convolutional neural networks in classification tasks [27]. Swin Transformer, based on ViT, optimized attention mechanisms and sampling effects, and learned the hierarchical structure of convolutional neural networks, significantly improving training speed and accuracy [28]. Researchers have applied it to the identification of forest fires, achieving rapid and real-time monitoring of forest fires [29].…”