Rethinking Transformer-based Set Prediction for Object Detection

Sun, Zhiqing; Cao, Shengcao; Yang, Yiming; Kitani, Kris M.

doi:10.1109/iccv48922.2021.00359

Cited by 236 publications

(87 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, to learn the global information, Wu et al [40] proposed a self-attention network for MR imaging and Feng et al [7] designed a transformer network for MR imaging. However, the selfattention [3,14,36] and transformer [21,33] have high computation complexity and occupy a huge number of GPU memory. Moreover, the transformer [33,35] is difficult to optimize and requires a large-scale training dataset.…”

Section: Mri Reconstructionmentioning

confidence: 99%

“…However, the selfattention [3,14,36] and transformer [21,33] have high computation complexity and occupy a huge number of GPU memory. Moreover, the transformer [33,35] is difficult to optimize and requires a large-scale training dataset. Different from them, in this work, the proposed Spatial and Fourier Layer (SFL) can simultaneously learn the local and global information of the feature, while the required memory and the computation complexity of SFL based on the Fast Fourier Transform are similar to the convolution layer.…”

Section: Mri Reconstructionmentioning

confidence: 99%

See 1 more Smart Citation

Contrastive Learning for Local and Global Learning MRI Reconstruction

Yi¹,

Liu²,

Le³

et al. 2021

Preprint

View full text Add to dashboard Cite

Magnetic Resonance Imaging (MRI) is an important medical imaging modality, while it requires a long acquisition time. To reduce the acquisition time, various methods have been proposed. However, these methods failed to reconstruct images with a clear structure for two main reasons. Firstly, similar patches widely exist in MR images, while most previous deep learning-based methods ignore this property and only adopt CNN to learn local information. Secondly, the existing methods only use clear images to constrain the upper bound of the solution space, while the lower bound is not constrained, so that a better parameter of the network cannot be obtained. To address these problems, we propose a Contrastive Learning for Local and Global Learning MRI Reconstruction Network (CLGNet). Specifically, according to the Fourier theory, each value in the Fourier domain is calculated from all the values in Spatial domain. Therefore, we propose a Spatial and Fourier Layer (SFL) to simultaneously learn the local and global information in Spatial and Fourier domains. Moreover, compared with self-attention and transformer, the SFL has a stronger learning ability and can achieve better performance in less time. Based on the SFL, we design a Spatial and Fourier Residual block as the main component of our model. Meanwhile, to constrain the lower bound and upper bound of the solution space, we introduce contrastive learning, which can pull the result closer to the clear image and push the result further away from the undersampled image. Extensive experimental results on different datasets and acceleration rates demonstrate that the proposed CLGNet achieves new state-of-the-art results.

show abstract

Section: Mri Reconstructionmentioning

confidence: 99%

Section: Mri Reconstructionmentioning

confidence: 99%

Contrastive Learning for Local and Global Learning MRI Reconstruction

Yi¹,

Liu²,

Le³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent works such as Swin Transformer [37], PVT [54] and CrossViT [5] improve architec-ture from different aspects. Furthermore, other researchers apply vision transformers to downstream tasks like semantic segmentation [60,62], object detection [3,51] and multimodal tasks [23]. Meanwhile, more detailed comparisons between vision Transformer and CNNs are investigated to show their relative strength and weakness [10,14,45,64].…”

Section: Vision Transformersmentioning

confidence: 99%

Exploiting Both Domain-specific and Invariant Knowledge via a Win-win Transformer for Unsupervised Domain Adaptation

Ma¹,

Zhang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Most existing UDA approaches enable knowledge transfer via learning domain-invariant representation and sharing one classifier across two domains. However, ignoring the domain-specific information that are related to the task, and forcing a unified classifier to fit both domains will limit the feature expressiveness in each domain. In this paper, by observing that the Transformer architecture with comparable parameters can generate more transferable representations than CNN counterparts, we propose a Win-Win TRansformer framework (WinTR) that separately explores the domain-specific knowledge for each domain and meanwhile interchanges cross-domain knowledge. Specifically, we learn two different mappings using two individual classification tokens in the Transformer, and design for each one a domain-specific classifier. The crossdomain knowledge is transferred via source guided label refinement and single-sided feature alignment with respect to source or target, which keeps the integrity of domainspecific information. Extensive experiments on three benchmark datasets show that our method outperforms the stateof-the-art UDA methods, validating the effectiveness of exploiting both domain-specific and invariant information for both domains in UDA.

show abstract

“…Then, the Vision Transformer (ViT) [35] utilized a pure Transformer framework to deal with vision tasks, treating an image as a collection of spatial patches. Recently, Transformers have achieved excellent outcomes on a variety of vision tasks [13-15, 42, 45, 46], including image recognition [15,[45][46][47][48], semantic segmentation [42], and object detection [13,14]. On semantic medical image segmentation, Transformer-combined architectures can be divided into two categories: the main one adopts self-attention like operations to complement CNNs [1,[49][50][51]; the other uses pure Transformers to constitute encoderdecoder architectures so as to capture deep representations and predict the class of each image pixel [42][43][44]53].…”

Section: Introductionmentioning

confidence: 99%

D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation

Wu¹,

Liao²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Computer-aided medical image segmentation has been applied widely in diagnosis and treatment to obtain clinically useful information of shapes and volumes of target organs and tissues. In the past several years, convolutional neural network (CNN) based methods (e.g., U-Net) have dominated this area, but still suffered from inadequate long-range information capturing. Hence, recent work presented computer vision Transformer variants for medical image segmentation tasks and obtained promising performances. Such Transformers model longrange dependency by computing pair-wise patch relations. However, they incur prohibitive computational costs, especially on 3D medical

show abstract

Rethinking Transformer-based Set Prediction for Object Detection

Cited by 236 publications

References 20 publications

Contrastive Learning for Local and Global Learning MRI Reconstruction

Contrastive Learning for Local and Global Learning MRI Reconstruction

Exploiting Both Domain-specific and Invariant Knowledge via a Win-win Transformer for Unsupervised Domain Adaptation

D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation

Contact Info

Product

Resources

About