Associative Domain Adaptation

Abstract

We propose associative domain adaptation, a novel technique for end-to-end domain adaptation with neural networks, the task of inferring class labels for an unlabeled target domain based on the statistical properties of a labeled source domain. Our training scheme follows the paradigm that in order to effectively derive class labels for the target domain, a network should produce statistically domain invariant embeddings, while minimizing the classification error on the labeled source domain. We accomplish this by reinforcing associations between source and target data directly in embedding space. Our method can easily be added to any existing classification network with no structural and almost no computational overhead. We demonstrate the effectiveness of our approach on various benchmarks and achieve state-of-the-art results across the board with a generic convolutional neural network architecture not specifically tuned to the respective tasks. Finally, we show that the proposed association loss produces embeddings that are more effective for domain adaptation compared to methods employing maximum mean discrepancy as a similarity measure in embedding space.

scAlign: a tool for alignment, integration and rare cell identification from scRNA-seq data

“…Viewed from the lens of perturbation studies, if sequencing a cell immediately before and after stimulus were possible, alignment would bring cells post-stimulus into the same region of alignment space as the cell before stimulus, therefore removing the effect of the stimulus. scAlign encodes the alignment space by extending the recent approach of learning by association for neural networks 49,50 into a unified framework for both unsupervised and supervised applications. For notational simplicity, we will assume we are aligning scRNA-seq data from a pair of conditions, though the framework extends to multiple conditions (see below).…”

Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

“…On par with these methods aligning distributions in the feature space, some methods align distributions in raw pixel space by translating source data to the target domain with Image to Image translation techniques [25][26][27][28][29][30]. In addition to domain-level distribution alignment, the class-level information in target data is also frequently used to align class-level distributions [11][12][13][14][15]31]. Compared with these methods, our method not only aligns class-level distributions, but also learns target discriminative features.…”

d-SNE: Domain Adaptation Using Stochastic Neighborhood Embedding

“…To this end, they added linear transformations between the weights to regularize the networks to behave thusly. Associative Domain Adaptation is another technique in the DMTL regime proposed by Haeusser et al which enforced association between the source and target domains [10]. CCSA and FADA furthered the contrastive loss techniques by creating a unified framework for supervised domain adaptation and gen-eralization [16,15].…”