Online Heterogeneous Transfer by Hedge Ensemble of Offline and Online Decisions

Abstract

In this paper, we study the online heterogeneous transfer (OHT) learning problem, where the target data of interest arrive in an online manner, while the source data and auxiliary co-occurrence data are from offline sources and can be easily annotated. OHT is very challenging, since the feature spaces of the source and target domains are different. To address this, we propose a novel technique called OHT by hedge ensemble by exploiting both offline knowledge and online knowledge of different domains. To this end, we build an offline decision function based on a heterogeneous similarity that is constructed using labeled source data and unlabeled auxiliary co-occurrence data. After that, an online decision function is learned from the target data. Last, we employ a hedge weighting strategy to combine the offline and online decision functions to exploit knowledge from the source and target domains of different feature spaces. We also provide a theoretical analysis regarding the mistake bounds of the proposed approach. Comprehensive experiments on three real-world data sets demonstrate the effectiveness of the proposed technique.

A survey on heterogeneous transfer learning

“…higher accuracy, on benchmark datasets which proves the method as an effective technique for knowledge transfer for online learning tasks. Yan et al [19] also proposed a method for online HTL learning tasks called Online Heterogeneous Transfer with Weighted Classifiers (OHTWC). This method proposes using unlabeled co-occurrence data to serve as a bridge between the two domains.…”

“…The larger the loss, the larger the weight decrease occurs to such classifier causing the more accurate classifier to have a higher weight for prediction purposes. Figure 2, adapted from [19], provides an illustration of this method for using weighted online and offline classifiers to perform an online task. An experiment for textto-image classification was performed to test the effectiveness of the proposed algorithm.…”

“…It aims to solve scenarios when the source and target domain are represented Fig. 2 Illustration of OHTWC adapted from [19] showing the use of offline and online classifiers to predict labels for an online task by heterogeneous features and dimensions. This method uses a labeled source with a limited labeled target along with a symmetric transformation for the feature space differences.…”

“…One of the patterns that can be noticed amongst them to accomplish this is the use of co-occurrence data. TTI [36], TLRisk [37], OHTWC [19], CT-Learn [57], and DCN [103] use co-occurrence data as a bridge between the source and target feature spaces. Co-occurrence data is most often used for text aided image classification and is relatively cheap to collect.…”

Online Bagging for Anytime Transfer Learning

“…The paper [22] uses ensemble learning and active learning to build a stable online learning framework to solve the problem drift problem in the online data flow. Yanet al [9] solves online heterogeneous transfer learning tasks by building a classifier by combining the weighted ensemble methods of offline and online decision making. In the paper [8], the traditional offline form of Boosting for Transfer Learning (TrAdaboost) is modified into an online transfer boosting method combined with the promotion method.…”