Transfer Learning across Feature-Rich Heterogeneous Feature Spaces via Feature-Space Remapping (FSR)

Abstract

Transfer learning aims to improve performance on a target task by utilizing previous knowledge learned from source tasks. In this paper we introduce a novel heterogeneous transfer learning technique, Feature- Space Remapping (FSR), which transfers knowledge between domains with different feature spaces. This is accomplished without requiring typical feature-feature, feature instance, or instance-instance co-occurrence data. Instead we relate features in different feature-spaces through the construction of meta-features. We show how these techniques can utilize multiple source datasets to construct an ensemble learner which further improves performance. We apply FSR to an activity recognition problem and a document classification problem. The ensemble technique is able to outperform all other baselines and even performs better than a classifier trained using a large amount of labeled data in the target domain. These problems are especially difficult because in addition to having different feature-spaces, the marginal probability distributions and the class labels are also different. This work extends the state of the art in transfer learning by considering large transfer across dramatically different spaces.

A survey of transfer learning

“…In Gong [42] and Blitzer [5], semi-supervised transfer learning is the case of having abundant labeled source data and limited labeled target data, and unsupervised transfer learning is the case of abundant labeled source data and no labeled target data. Cook [19] and Feuz [36] provide a different variation where the definition of supervised or unsupervised refers to the presence or absence of labeled data in the source domain and informed or uninformed refers to the presence or absence of labeled data in the target domain. With this definition, a labeled source and limited labeled target domain is referred to as informed supervised transfer learning.…”

Data volume reduction in covering approximation spaces with respect to twenty-two types of covering based rough sets

“…Possible future research directions include the application of consistent mappings in transfer learning problems like [54] as well as adopting the proposed algorithms to Big data computing paradigms like Mapreduce [55]. From a theoretical point of view, it is interesting to find necessary and sufficient condition(s) for a mapping to be a homomorphism or isomorphism in the sense of Definitions 3.2 and 3.1.…”

A survey on heterogeneous transfer learning

“…For example, Liu et al [11] use the notation of supervised transfer learning to denote a fully-labeled source domain and a limited labeled target, while unsupervised transfer learning denotes a mostly labeled source with no target domain labels. Conversely, the work of Cook and Feuz [12] who use supervised and unsupervised transfer learning to denote the presence or absence of labels respectively only in the source domain. Furthermore, they use the notation of informed and uninformed to denote the presence or absence of labels respectively in the target domain.…”

“…Feuz and Cook [12] developed three variants of their proposed Feature-Space Remapping (FSR) method for HTL tasks when one has either limited target labels or optionally no target labels as well as an ensemble technique. This work extends from the previous version of their paper [89].…”

“…When using a heterogeneous auxiliary dataset, the class labels may not match in real-world scenarios. Specifically, FSR [12], HeMap [26], and Proactive HTL [54] are the only surveyed methods that directly address this issue of differing label spaces. Also, when drawing data from a different domain, this auxiliary data may exhibit data distribution differences and further adaptation is required in this case to correct this after resolving the feature space gap.…”