AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Data

Abstract

We provide a regularization framework for subject transfer learning in which we seek to train an encoder and classifier to minimize classification loss, subject to a penalty measuring independence between the latent representation and the subject label. We introduce three notions of independence and corresponding penalty terms using mutual information or divergence as a proxy for independence. For each penalty term, we provide several concrete estimation algorithms, using analytic methods as well as neural critic functions. We provide a hands-off strategy for applying this diverse family of regularization algorithms to a new dataset, which we call "AutoTransfer". We evaluate the performance of these individual regularization strategies and our AutoTransfer method on EEG, EMG, and ECoG datasets, showing that these approaches can improve subject transfer learning for challenging real-world datasets.

AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Data

“…This bound will be tight for an adversary whose predicted distribution over subjects is close to the true posterior distribution; thus we can improve the quality of this surrogate objective by using a strong adversary model that is trained to convergence. See Appendix A1 of [8] for further details.…”

“…We explore other kernel-based score function estimation methods based on the work of Zhou, Shi, and Zhu [11], who frame the problem of score function estimation as a regularized vector regression problem. See Appendix A2d of [8] for further details about the estimators used and how their hyperparameters were set.…”

“…Gretton, Borgwardt, Rasch, et al [6] derive a kernel estimate of the MMD using a radial basis function kernel (see details in Appendix B1 of [8]). We use their empirical estimate, with a kernel length scale set by the median heuristic [12] as we do for a subset of MIGE experiments.…”

“…, M } controlling the number of nuisance values included, and compute a term for all combinations within this subset, which we call a "clique" subset selection. These two selection procedures are described in more detail in Appendix B2 of [8].…”

“…In this section, we derive several concrete methods for estimating the mutual information and divergence penalties used in the regularized objective functions outlined above. Further details including pseudocode for the marginal, conditional, and complementary versions of each technique can be found starting in Appendix A1 of [8].…”