Rethinking Importance Weighting for Transfer Learning

Abstract

A key assumption in supervised learning is that training and test data follow the same probability distribution. However, this fundamental assumption is not always satisfied in practice, e.g., due to changing environments, sample selection bias, privacy concerns, or high labeling costs. Transfer learning (TL) relaxes this assumption and allows us to learn under distribution shift. Classical TL methods typically rely on importance-weighting-a predictor is trained based on the training losses weighted according to the importance (i.e., the testover-training density ratio). However, as real-world machine learning tasks are becoming increasingly complex, high-dimensional, and dynamical, novel approaches are explored to cope with such challenges recently. In this article, after introducing the foundation of TL based on importance-weighting, we review recent advances based on joint and dynamic importancepredictor estimation. Furthermore, we introduce a method of causal mechanism transfer that incorporates causal structure in TL. Finally, we discuss future perspectives of TL research.

Learning under Data Drift with Time-Varying Importance Weights

“…Given its importance in real-world applications, the problem of how to learn from shifting distributions has been widely studied. Much past work has focused on a single shift between training/test data (Lu et al, 2021;Wang & Deng, 2018;Fakoor et al, 2020b) as well as restricted forms of shift involving changes in only the features (Sugiyama et al, 2007a;Reddi et al, 2015a), labels (Lipton et al, 2018;Garg et al, 2020;Alexandari et al, 2020), or in the underlying relationship between the two (Zhang et al, 2013;Lu et al, 2018). Past approaches to handle distributions evolving over time have been considered in the literature on: concept drift Gomes et al ( 2019); Souza et al (2020), reinforcement learning (shift between the target policy and behavior policy) Schulman et al (2015); Wang et al (2016);Fakoor et al (2020a), (meta) online learning Shalev-Shwartz (2012); Finn et al (2019); Harrison et al (2020); Wu et al (2021), and task-free continual/incremental learning Aljundi et al (2019); He et al (2019), but to our knowledge, existing methods for these settings do not employ time-varying data weights like we propose here.…”