A new topic-bridged model for transfer learning

Wu, Meng-Sung; Chien, Jen‐Tzung

doi:10.1109/icassp.2010.5494947

Cited by 5 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to fully utilize the knowledge of the source domain, additional penalty terms are added for must-link and cannot-link constraints. An extension of PLSA (a) Topic-bridged PLSA (Xue et al, 2008) and Topic-bridged LDA (Wu & Chien, 2010) (named Dual-PLSA (Yoo & Choi, 2009;Gao & Li, 2011) (Gao & Li, 2011) is proposed to mine the topics shared by two domains, where topics are defined as word-pair distributions rather than word distributions which models the cross-domain word co-occurrence relations. The aforementioned models all try to learn the shared topics between domains but the domain-specific properties are ignored, and irrelevant topics may degrade the task performance in the target domain.…”

Section: Share Topicmentioning

confidence: 99%

“…For example, Short Text Similarity (STS) evaluates the semantic similarity between two short text snippets (target domain). Since they are short (only a few sentences, e.g., a tweet), standard statistical Bayesian linear and logistic regression (Friedman, Hastie, & Tibshirani, 2001) (Sultan et al, 2016) Probabilistic matrix factorization model (PMF) (Mnih & Salakhutdinov, 2008) (Jing et al, 2014) (Iwata & Koh, 2015) Flexible mixture model (Si & Jin, 2003) (Li et al, 2009) Polylingual topic models (Mimno, Wallach, Naradowsky, Smith, & McCallum, 2009) (Hu et al, 2014) Probabilistic latent semantic analysis (PLSA) (Hofmann, 1999) (Xue et al, 2008), (Gao & Li, 2011), (Zhuang et al, 2013), (Zhuang et al, 2010), (Zhuang et al, 2012), (Li et al, 2012), (Zhai et al, 2004(Zhai et al, ), et al, 2009 Latent Dirichlet allocation (LDA) (Blei et al, 2003) (Wu & Chien, 2010), (Jin et al, 2011), (Chen et al, 2015), (Yu & Aloimonos, 2010), (Yang et al, 2011), (Tang et al, 2012), (Phan et al, 2011) Probabilistic linear discriminant analysis (PLDA) (Prince & Elder, 2007) (Hong, Zhang, Li, Wan, & Tong, 2016) (López & Lleida, 2012) Conditional random field (CRF) (Lafferty et al, 2001) (Nallapati, Surdeanu, & Manning, 2010) (Finkel & Manning, 2009) (Arnold et al, 2008) Hierarchal latent Dirichlet allocation (hLDA)…”

Section: Natural Language Processingmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Transfer Learning: An Overview of Probabilistic Graphical Models for Transfer Learning

Xuan,

Lu,

Zhang

2021

Preprint

View full text Add to dashboard Cite

Transfer learning where the behavior of extracting transferable knowledge from the source domain(s) and reusing this knowledge to target domain has become a research area of great interest in the field of artificial intelligence. Probabilistic graphical models (PGMs) have been recognized as a powerful tool for modeling complex systems with many advantages, e.g., the ability to handle uncertainty and possessing good interpretability. Considering the success of these two aforementioned research areas, it seems natural to apply PGMs to transfer learning. However, although there are already some excellent PGMs specific to transfer learning in the literature, the potential of PGMs for this problem is still grossly underestimated. This paper aims to boost the development of PGMs for transfer learning by 1) examining the pilot studies on PGMs specific to transfer learning, i.e., analyzing and summarizing the existing mechanisms particularly designed for knowledge transfer; 2) discussing examples of real-world transfer problems where existing PGMs have been successfully applied; and 3) exploring several potential research directions on transfer learning using PGM.

show abstract

Section: Share Topicmentioning

confidence: 99%

Section: Natural Language Processingmentioning

confidence: 99%

Bayesian Transfer Learning: An Overview of Probabilistic Graphical Models for Transfer Learning

Xuan,

Lu,

Zhang

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Datasets structured like this were first used by Dai et al [13,14] to test two different approaches: CoCC [13], which co-clusters domains and words as a means to propagate the class structure from the source domain to the target domain; and TrAdaBoost [14], an extension of AdaBoost that implements transfer learning. Since then, many authors have adopted experimental settings with the same structure, in order to test transfer learning systems based on topic models (e.g., Topic-Bridged PLSA (TPLSA - [60]), Topic-Bridged LDA (TLDA - [55]), and Partially Supervised Cross-Collection LDA (PSCCLDA - [4])), non-negative matrix factorization (e.g., MTrick [65]), probabilistic models (e.g., Topic Correlation Analysis (TCA - [27])), and clustering techniques (e.g., Cross-Domain Spectral Classification (CDSC - [30])).…”

Section: Transductive Transfer Problemsmentioning

confidence: 99%

“…However, although these methods have been tested on transductive transfer problems (i.e., by having A T B T play the role of Ob U T and T e U T at the same time), not all of them are transductive transfer methods as defined in Section 2. Indeed, TrAdaBoost [14], TLDA [55], and TCA [27] are inductive transfer methods; i.e., when applied to a transductive problem, a "TTLP-via-ITLM approach" must be followed. When inductive transfer learning methods are tested on an inductive transfer learning problem, they are meant to be tested on a test set T e U T different from the unlabelled set T r U T on which they have been trained, in order to show that they generalize.…”

Section: Transductive Transfer Problemsmentioning

confidence: 99%

Lost in Transduction: Transductive Transfer Learning in Text Classification

Moreo

Esuli

Sebastiani

2021

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

Obtaining high-quality labelled data for training a classifier in a new application domain is often costly. Transfer Learning (a.k.a. “Inductive Transfer”) tries to alleviate these costs by transferring, to the “target” domain of interest, knowledge available from a different “source” domain. In transfer learning the lack of labelled information from the target domain is compensated by the availability at training time of a set of unlabelled examples from the target distribution. Transductive Transfer Learning denotes the transfer learning setting in which the only set of target documents that we are interested in classifying is known and available at training time. Although this definition is indeed in line with Vapnik’s original definition of “transduction”, current terminology in the field is confused. In this article, we discuss how the term “transduction” has been misused in the transfer learning literature, and propose a clarification consistent with the original characterization of this term given by Vapnik. We go on to observe that the above terminology misuse has brought about misleading experimental comparisons, with inductive transfer learning methods that have been incorrectly compared with transductive transfer learning methods. We then, give empirical evidence that the difference in performance between the inductive version and the transductive version of a transfer learning method can indeed be statistically significant (i.e., that knowing at training time the only data one needs to classify indeed gives an advantage). Our clarification allows a reassessment of the field, and of the relative merits of the major, state-of-the-art algorithms for transfer learning in text classification.

show abstract

“…They make use of TF-IDF ranking technique in construction of the user profile, which they use for recommending other Twitter users to follow. c) Micro-post Classification: In [12], [26] the authors present LDA transfer learning. Transfer Learning is the process of generic learning in one domain and applying the model in a different domain.…”

Section: Tweet Message Classificationmentioning

confidence: 99%

Entity disambiguation in tweets leveraging user social profiles

Yerva

Catasta

Demartini

et al. 2013

2013 IEEE 14th International Conference on Information Reuse &Amp; Integration (IRI)

View full text Add to dashboard Cite

Abstract-Pervasive web and social networks are becoming part of everyone's life. Users through their activities on these networks are leaving traces of their expertise, interests and personalities. With the advances in Web mining and user modeling techniques it is possible to leverage the user social network activity history to extract the semantics of user-generated content. In this work we explore various techniques for constructing user profiles based on the content they publish on social networks. We further show that one of the advantages of maintaining social network user profiles is to provide the context for better understanding of microposts. We propose and experimentally evaluate different approaches for entity disambiguation in social networks based on syntactic and semantic features on top of two different social networks: a general-interest network (i.e., Twitter) and a domain-specific network (i.e., StackOverflow). We demonstrate how disambiguation accuracy increases when considering enriched user profiles integrating content from both social networks.

show abstract

A new topic-bridged model for transfer learning

Cited by 5 publications

References 9 publications

Bayesian Transfer Learning: An Overview of Probabilistic Graphical Models for Transfer Learning

Bayesian Transfer Learning: An Overview of Probabilistic Graphical Models for Transfer Learning

Lost in Transduction: Transductive Transfer Learning in Text Classification

Entity disambiguation in tweets leveraging user social profiles

Contact Info

Product

Resources

About