Contrastive Fine-grained Class Clustering via Generative Adversarial Networks

Kim, Yun-Ji; Ha, Jung-Woo

doi:10.48550/arxiv.2112.14971

Cited by 4 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a complete picture of the field, readers may refer to the survey by Min et al (2018). We emphasize deep-clustering-based approaches, which attempt to learn the feature representation of the data while simultaneously discovering the underlying clusters: K-means Caron et al, 2018), information maximization (Menapace et al, 2020;Ji et al, 2019;Kim and Ha, 2021;Do et al, 2021), transport alignment (Asano et al, 2019;Caron et al, 2020;Wang et al, 2022), neighborhood-clustering (Xie et al, 2016;Huang et al, 2019;Dang et al, 2021), contrastive learning (Pan and Kang, 2021;Shen et al, 2021), probabilistic approaches Monnier et al, 2020;Falck et al, 2021;Manduchi et al, 2021), and kernel density (Yang and Li, 2021). These works primarily focus on clustering data for downstream tasks for a single domain, whereas our clustering algorithm is designed to cluster the data from multiple domains.…”

Section: Related Workmentioning

confidence: 99%

A Prototype-Oriented Clustering for Domain Shift with Source Privacy

Tanwisuth¹,

Zhang²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

Unsupervised clustering under domain shift (UCDS) studies how to transfer the knowledge from abundant unlabeled data from multiple source domains to learn the representation of the unlabeled data in a target domain. In this paper, we introduce Prototype-oriented Clustering with Distillation (PCD) to not only improve the performance and applicability of existing methods for UCDS, but also address the concerns on protecting the privacy of both the data and model of the source domains. PCD first constructs a source clustering model by aligning the distributions of prototypes and data. It then distills the knowledge to the target model through cluster labels provided by the source model while simultaneously clustering the target data. Finally, it refines the target model on the target domain data without guidance from the source model. Experiments across multiple benchmarks show the effectiveness and generalizability of our source-private clustering method.

show abstract

Section: Related Workmentioning

confidence: 99%

A Prototype-Oriented Clustering for Domain Shift with Source Privacy

Tanwisuth¹,

Zhang²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…SSL can be especially useful for tasks requiring heavy annotation costs, such as finegrained image recognition [7], because it aims to learn discriminative representations without using human annotation. However, there are several limitations to the application of current SSL methods for fine-grained image recognition tasks [8][9][10]. In contrast with ordinary computer vision tasks, fine-grained images share many visual characteristics across their classes.…”

Section: Introductionmentioning

confidence: 99%

Patch-Level Consistency Regularization in Self-Supervised Transfer Learning for Fine-Grained Image Recognition

Lee,

Hwang

2023

Applied Sciences

View full text Add to dashboard Cite

Fine-grained image recognition aims to classify fine subcategories belonging to the same parent category, such as vehicle model or bird species classification. This is an inherently challenging task because a classifier must capture subtle interclass differences under large intraclass variances. Most previous approaches are based on supervised learning, which requires a large-scale labeled dataset. However, such large-scale annotated datasets for fine-grained image recognition are difficult to collect because they generally require domain expertise during the labeling process. In this study, we propose a self-supervised transfer learning method based on Vision Transformer (ViT) to learn finer representations without human annotations. Interestingly, it is observed that existing self-supervised learning methods using ViT (e.g., DINO) show poor patch-level semantic consistency, which may be detrimental to learning finer representations. Motivated by this observation, we propose a consistency loss function that encourages patch embeddings of the overlapping area between two augmented views to be similar to each other during self-supervised learning on fine-grained datasets. In addition, we explore effective transfer learning strategies to fully leverage existing self-supervised models trained on large-scale labeled datasets. Contrary to the previous literature, our findings indicate that training only the last block of ViT is effective for self-supervised transfer learning. We demonstrate the effectiveness of our proposed approach through extensive experiments using six fine-grained image classification benchmark datasets, including FGVC Aircraft, CUB-200-2011, Food-101, Oxford 102 Flowers, Stanford Cars, and Stanford Dogs. Under the linear evaluation protocol, our method achieves an average accuracy of 78.5%, outperforming the existing transfer learning method, which yields 77.2%.

show abstract

“…While the explicit function of these networks is to generate artificial examples of some set of input data, training these networks aims to learn the statistical distribution of the input data in a multidimensional parameter space (Creswell et al 2018). As a result, GANs can be useful for any task where identifying this distribution is advantageous and can be exploited, such as data augmentation (Bousmalis et al 2017), classification (Radford et al 2015), clustering (Kim and Ha 2021), or the transfer of properties from one dataset onto another (i.e. style transfer; Karras et al 2018).…”

Section: Introductionmentioning

confidence: 99%

This population does not exist: learning the distribution of evolutionary histories with generative adversarial networks

Booker

Ray

Schrider

2022

Preprint

View full text Add to dashboard Cite

Numerous studies over the last decade have demonstrated the utility of machine learning methods when applied to population genetic tasks. More recent studies show the potential of deep learning methods in particular, which allow researchers to approach problems without making prior assumptions about how the data should be summarized or manipulated, instead learning their own internal representation of the data in an attempt to maximize inferential accuracy. One type of deep neural network, called Generative Adversarial Networks (GANs), can even be used to generate new data, and this approach has been used to create individual artificial human genomes free from privacy concerns. In this study, we further explore the application of GANs in population genetics by designing and training a network to learn the statistical distribution of population genetic alignments (i.e. data sets consisting of sequences from an entire population sample) under several diverse evolutionary histories—the first GAN capable of performing this task. After testing multiple different neural network architectures, we report the results of a fully differentiable Deep-Convolutional Wasserstein GAN with gradient penalty that is capable of generating artificial examples of population genetic alignments that successfully mimic key aspects of the training data, including the site frequency spectrum, differentiation between populations, and patterns of linkage disequilibrium. We demonstrate consistent training success across various evolutionary models, including models of panmictic and subdivided populations, populations at equilibrium and experiencing changes in size, and populations experiencing either no selection or positive selection of various strengths, all without the need for extensive hyperparameter tuning. Overall, our findings highlight the ability of GANs to learn and mimic population genetic data, and suggest future areas where this work can be applied in population genetics research that we discuss herein.AUTHOR SUMMARYThe application of deep-learning to biological problems has expanded greatly over the last decade. One type of deep neural network, called a Generative Adversarial Network (GAN), attempts to generate artificial examples of a given type of data by learning to fool a discriminator that is simultaneously learning to discriminate between real and artificial examples. In this study, we design a GAN whose purpose is to generate artificial examples of genetic alignments from biological populations of varying evolutionary histories—essentially learning the statistical distribution of those evolutionary histories. We show that our GAN is able to mimic key aspects of the genetic alignments relevant to population genetics, and that the GAN does not require extensive tuning of the network parameters. Ultimately, this work demonstrates the ability of these networks to learn and mimic population genetic data, and highlights future areas where this work can be applied and expanded.

show abstract

Contrastive Fine-grained Class Clustering via Generative Adversarial Networks

Cited by 4 publications

References 16 publications

A Prototype-Oriented Clustering for Domain Shift with Source Privacy

A Prototype-Oriented Clustering for Domain Shift with Source Privacy

Patch-Level Consistency Regularization in Self-Supervised Transfer Learning for Fine-Grained Image Recognition

This population does not exist: learning the distribution of evolutionary histories with generative adversarial networks

Contact Info

Product

Resources

About