Multi-manifold Attention for Vision Transformers

Konstantinidis, Dimitrios; Papastratis, Ilias; Dimitropoulos, Kosmas; Daras, Petros

doi:10.48550/arxiv.2207.08569

Cited by 1 publication

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Manifold Learning. Recently, manifolds learning has been widely employed in vision tasks (Konstantinidis et al 2022;Luo et al 2020;Huang et al 2017), largely attributes to…”

Section: Related Workmentioning

confidence: 99%

“…Based on the SPD representation setting (Konstantinidis et al 2022), it is composed of square matrices M of size d × d, which can be denoted as:…”

Section: Inter-student Knowledge Transfermentioning

confidence: 99%

“…The proposed method can be learned synergistically with CNN and Transformer to adaptively learn manifolds of varying structure across samples. There have been some methods (Konstantinidis et al 2022;Huang et al 2017) employing manifolds to prove that features in different manifolds carry special statistical and geometrical properties, which bring complementary discriminative power for various tasks. Recent advances in manifold learning reveal two properties (Konstantinidis et al 2022): (1) allowing similar features to appear closer to each other, while dissimilar features move further apart; (2) providing superiors manifold metric to measure the discrepancy with different statistical and geometrical properties.…”

Section: Introductionmentioning

confidence: 99%

“…There have been some methods (Konstantinidis et al 2022;Huang et al 2017) employing manifolds to prove that features in different manifolds carry special statistical and geometrical properties, which bring complementary discriminative power for various tasks. Recent advances in manifold learning reveal two properties (Konstantinidis et al 2022): (1) allowing similar features to appear closer to each other, while dissimilar features move further apart; (2) providing superiors manifold metric to measure the discrepancy with different statistical and geometrical properties. Inspired by these attempts, M-CnT incorporates more compact and discriminative embeddings in the manifold space considering that: (1) Intrastudent class-wise consistency: to strengthen the discriminative power of both CNN and Transformer, the intra-class samples are required to be compact, while the inter-class samples are separable.…”

Section: Introductionmentioning

confidence: 99%

“…(2) Inter-student knowledge transfer: CNN and Transformer have distinct inner feature flow forms, in which their features with complementary classwise distribution create a potential opportunity for collaboration. In that, the inter-student discrepancy is represented as the distance between two submanifolds, which is then minimized based on the defined manifold metric (e.g., symmetric positive definite metric (Konstantinidis et al 2022)).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Huang,

Xie

et al. 2024

AAAI

View full text Add to dashboard Cite

Semi-supervised learning (SSL), as one of the dominant methods, aims at leveraging the unlabeled data to deal with the annotation dilemma of supervised learning, which has attracted much attentions in the medical image segmentation. Most of the existing approaches leverage a unitary network by convolutional neural networks (CNNs) with compulsory consistency of the predictions through small perturbations applied to inputs or models. The penalties of such a learning paradigm are that (1) CNN-based models place severe limitations on global learning; (2) rich and diverse class-level distributions are inhibited. In this paper, we present a novel CNN-Transformer learning framework in the manifold space for semi-supervised medical image segmentation. First, at intra-student level, we propose a novel class-wise consistency loss to facilitate the learning of both discriminative and compact target feature representations. Then, at inter-student level, we align the CNN and Transformer features using a prototype-based optimal transport method. Extensive experiments show that our method outperforms previous state-of-the-art methods on three public medical image segmentation benchmarks.

show abstract

“…Manifold Learning. Recently, manifolds learning has been widely employed in vision tasks (Konstantinidis et al 2022;Luo et al 2020;Huang et al 2017), largely attributes to…”

Section: Related Workmentioning

confidence: 99%

“…Based on the SPD representation setting (Konstantinidis et al 2022), it is composed of square matrices M of size d × d, which can be denoted as:…”

Section: Inter-student Knowledge Transfermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Huang,

Xie

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

Multi-manifold Attention for Vision Transformers

Cited by 1 publication

References 0 publications

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Contact Info

Product

Resources

About