On Universal Transfer Learning

Mahmud, Maqsood

doi:10.1007/978-3-540-75225-7_14

Cited by 14 publications

(14 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training for each data set in every case was done using the training set, while the results reported are all for performance on the testing set. We also performed 135 other transfer experiments using other combinations of the percent of data used as training and testing sets [29]. The results of these experiments also, by and large, exhibit the same properties as the results in this paper, which we now discuss and interpret.…”

Section: Setup Of the Experimentssupporting

confidence: 69%

“…where the constant of inequality now depends only on U. Similar comments also apply to Theorem 3.9 and we refer the interested reader to [29] for a fuller development of the above.…”

Section: Competitive Optimality Of Universal Priorsmentioning

confidence: 79%

“…The arbitrary choice of data sets make our experiments the most general transfer experiments to date. This batch of experiments is in fact part of a larger set of 144 transfer experiments with similar results [29].…”

Section: Practical Approximationsmentioning

confidence: 98%

“…However our analysis shows that neither of these two are interesting -the first turns out to be merely single task learning in a product space and the second is more or less the same as sequential transfer in the general case. So at this juncture we do not expound on this further and refer the curious reader to [29] for details.…”

Section: Parallel Transfermentioning

confidence: 99%

See 3 more Smart Citations

On universal transfer learning

Mahmud

2009

Theoretical Computer Science

View full text Add to dashboard Cite

In transfer learning the aim is to solve new learning tasks using fewer examples by using information gained from solving related tasks. Existing transfer learning methods have been used successfully in practice and PAC analysis of these methods have been developed. But the key notion of relatedness between tasks has not yet been defined clearly, which makes it difficult to understand, let alone answer, questions that naturally arise in the context of transfer, such as, how much information to transfer, whether to transfer information, and how to transfer information across tasks. In this paper, we look at transfer learning from the perspective of Algorithmic Information Theory/Kolmogorov complexity theory, and formally solve these problems in the same sense Solomonoff Induction solves the problem of inductive inference. We define universal measures of relatedness between tasks, and use these measures to develop universally optimal Bayesian transfer learning methods. We also derive results in AIT that are interesting by themselves. To address a concern that arises from the theory, we also briefly look at the notion of Kolmogorov complexity of probability measures. Finally, we present a simple practical approximation to the theory to do transfer learning and show that even these are quite effective, allowing us to transfer across tasks that are superficially unrelated. The latter is an experimental feat which has not been achieved before, and thus shows the theory is also useful in constructing practical transfer algorithms.

show abstract

Section: Setup Of the Experimentssupporting

confidence: 69%

“…where the constant of inequality now depends only on U. Similar comments also apply to Theorem 3.9 and we refer the interested reader to [29] for a fuller development of the above.…”

Section: Competitive Optimality Of Universal Priorsmentioning

confidence: 79%

Section: Practical Approximationsmentioning

confidence: 98%

Section: Parallel Transfermentioning

confidence: 99%

See 2 more Smart Citations

On universal transfer learning

Mahmud

2009

Theoretical Computer Science

View full text Add to dashboard Cite

show abstract

“…Ling et al [12] developed a new spectral classification algorithm that optimized an objective function to seek for the maximal consistency between the supervised information from the source domain and the intrinsic structure of the target domain. Mahmud et al [13] studied transfer learning from the perspective of algorithmic information theory. They measured the relatedness between tasks, and then decided how much information to transfer and how to transfer.…”

Section: Computer Science and Technologymentioning

confidence: 99%

Inductive transfer learning for unlabeled target-domain via hybrid regularization

Zhuang

Luo²,

et al. 2009

Chin. Sci. Bull.

View full text Add to dashboard Cite

Citation: Zhuang F Z, Luo P, He Q, et al. Inductive transfer learning for unlabeled target-domain via hybrid regularization.Recent years have witnessed an increasing interest in transfer learning. This paper deals with the classification problem that the target-domain with a different distribution from the source-domain is totally unlabeled, and aims to build an inductive model for unseen data. Firstly, we analyze the problem of class ratio drift in the previous work of transductive transfer learning, and propose to use a normalization method to move towards the desired class ratio. Furthermore, we develop a hybrid regularization framework for inductive transfer learning. It considers three factors, including the distribution geometry of the target-domain by manifold regularization, the entropy value of prediction probability by entropy regularization, and the class prior by expectation regularization. This framework is used to adapt the inductive model learnt from the source-domain to the target-domain. Finally, the experiments on the real-world text data show the effectiveness of our inductive method of transfer learning. Meanwhile, it can handle unseen test points.

show abstract