Charting the Right Manifold: Manifold Mixup for Few-shot Learning

Mangla, Puneet; Singh, Mayank; Sinha, Abhishek; Kumari, Nupur; Balasubramanian, Vineeth N; Krishnamurthy, Balaji

doi:10.48550/arxiv.1907.12087

Cited by 4 publications

(17 citation statements)

References 39 publications

(64 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some meta learning models need to pretrain on a lager task of N-way K-shot before training on 5-way 5-shot(1-shot), which is called meta pretraining (Snell, Swersky, and Zemel 2017). Moreover, some models use self-supervised pretraining (Mangla et al 2019) or pertrained feature extractor (Lee et al 2019). However, our framework can be meta-trained end-to-end without any method of pretraining.…”

Section: Resultsmentioning

confidence: 99%

Region Comparison Network for Interpretable Few-shot Image Classification

Xue,

Duan,

et al. 2020

Preprint

View full text Add to dashboard Cite

While deep learning has been successfully applied to many real-world computer vision tasks, training robust classifiers usually requires a large amount of welllabeled data. However, the annotation is often expensive and time-consuming. Few-shot image classification has thus been proposed to effectively use only a limited number of labeled examples to train models for new classes. Recent works based on transferable metric learning methods have achieved promising classification performance through learning the similarity between the features of samples from the query and support sets. However, rare of them explicitly consider the model interpretability. For that, in this work, we propose a metric learning based method named Region Comparison Network (RCN), which aims to reveal how fewshot learning works as in a neural network, to learn specific regions that are related to each other in images coming from the query and support sets. Moreover, we design a visualization strategy named Region Activation Mapping (RAM) to intuitively explain what our method has learned by visualizing intermediate variables in our network. We also present a new way to generalize the interpretability from the task level to the category level, which can also be viewed as a way to find the prototypical parts for supporting the final decision of our RCN. Extensive experiments on four benchmark datasets clearly show the effectiveness of our method over existing baselines.

show abstract

Section: Resultsmentioning

confidence: 99%

Region Comparison Network for Interpretable Few-shot Image Classification

Xue,

Duan,

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The Nesterov Momentum optimizer [50] is used with an initial learning rate of 0.01. The total training epochs on the CUB, the miniImageNet and the Kinetics are 57, 40 and 23, and the learning rate is dropped to 10% on (30,40), (30,37), (2,19) epochs respectively. The weight decay is set to be 0.0005.…”

Section: Methodsmentioning

confidence: 99%

“…Self-supervised learning aims at learning from the supervision of the object structure, alleviating the need of supervision from manual labels, and has been researched in the field of unsupervised/semisupervised learning [31,42]. Recently, this mechanism is also applied in FSL by methods such as predicting the rotation [19,37] and predicting the relative position [19]. In this work, inspired by these previous works, we propose to use the self-supervised split loss to learn part-related primitives and alleviate the influence of the semantic gap between known and novel classes.…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, as objects with relatively large semantic gaps may share similar structures (e.g., the upper and lower parts of dogs and cars can be easily distinguished, although they are not highly semantically related), self-supervision from object structures [31,42] may help to alleviate the influence of semantic gaps. Therefore, inspired by current works [19,31,37] on self-supervised learning, we propose to use the split-based self-supervised mechanism for FSL, where we split the input image horizontally and vertically, perm the splits, and ask the model to recognize which perms are applied to them. Specifically, given an input image x, we first divide it along rows and columns into h • v splits.…”

Section: Primitive Discoverymentioning

confidence: 99%

“…The weight for this loss is typically 0.1. To learn the object structure implicitly, we also add the self-supervised rotation loss [19,37] to assist the learning. Given an input image, it is rotated by {0, 90, 180, 270} degrees, and the model is asked to predict which rotation is applied to the image.…”

Section: Auxiliary Objective and Regularizationmentioning

confidence: 99%

See 2 more Smart Citations

Compositional Few-Shot Recognition with Primitive Discovery and Enhancing

Zou

Zhang

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Few-shot learning (FSL) aims at recognizing novel classes given only few training samples, which still remains a great challenge for deep learning. However, humans can easily recognize novel classes with only few samples. A key component of such ability is the compositional recognition that human can perform, which has been well studied in cognitive science but is not well explored in FSL. Inspired by such capability of humans, to imitate humans' ability of learning visual primitives and composing primitives to recognize novel classes, we propose an approach to FSL to learn a feature representation composed of important primitives, which is jointly trained with two parts, i.e. primitive discovery and primitive enhancing. In primitive discovery, we focus on learning primitives related to object parts by self-supervision from the order of image splits, avoiding extra laborious annotations and alleviating the effect of semantic gaps. In primitive enhancing, inspired by current studies on the interpretability of deep networks, we provide our composition view for the FSL baseline model. To modify this model for effective composition, inspired by both mathematical deduction and biological studies (the Hebbian Learning rule and the Winner-Take-All mechanism), we propose a soft composition mechanism by enlarging the activation of important primitives while reducing that of others, so as to enhance the influence of important primitives and better utilize these primitives to compose novel classes. Extensive experiments on public benchmarks are conducted on both the few-shot image classification and video recognition tasks. Our method achieves the state-of-the-art performance on all these datasets and shows better interpretability. CCS CONCEPTS• Computing methodologies → Artificial intelligence.

show abstract

Embedding Propagation: Smoother Manifold for Few-Shot Classification

Rodríguez¹,

Laradji²,

Drouin³

et al. 2020

Preprint

View full text Add to dashboard Cite

Few-shot classification is challenging because the data distribution of the training set can be widely different to the one of test set as their classes are disjoint. This distribution shift often results in poor generalization. Manifold smoothing has been shown to address the distribution shift problem by extending the decision boundaries and reducing the noise of the class representations. Moreover, manifold smoothness is a key factor for semi-supervised learning and transductive learning algorithms. In this work, we present embedding propagation as an unsupervised nonparametric regularizer for manifold smoothing. Embedding propagation leverages interpolations between the extracted features of a neural network based on a similarity graph. We empirically show that embedding propagation yields a smoother embedding manifold. We also show that incorporating embedding propagation to a transductive classifier leads to new state-of-the-art results in miniImagenet, tiered Imagenet, and CUB. Furthermore, we show that embedding propagation results in additional improvement in performance for semi-supervised learning scenarios.

show abstract

Charting the Right Manifold: Manifold Mixup for Few-shot Learning

Cited by 4 publications

References 39 publications

Region Comparison Network for Interpretable Few-shot Image Classification

Region Comparison Network for Interpretable Few-shot Image Classification

Compositional Few-Shot Recognition with Primitive Discovery and Enhancing

Embedding Propagation: Smoother Manifold for Few-Shot Classification

Contact Info

Product

Resources

About