Learning to Predict Distributions of Words Across Domains

Bollegala, Danushka; Weir, David; Carroll, John M.

doi:10.3115/v1/p14-1058

Cited by 15 publications

(12 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subword approaches, on the other hand, are often more compositional and flexible, and we leave the extension of our method to handle subword information to future work. Our work is also related to some methods in domain adaptation and multi-lingual correlation, such as that of Bollegala et al (2014).…”

Section: Related Workmentioning

confidence: 99%

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

Khodak¹,

Saunshi²,

Liang³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

110

View full text Add to dashboard Cite

Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introducesà la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable "on the fly" in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how theà la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

Khodak¹,

Saunshi²,

Liang³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

110

View full text Add to dashboard Cite

show abstract

“…In contrast, in the movie domain, the word "lightweight" usually connotes a negative opinion describing movies that do not invoke deep thoughts among the audience. This observation motivates the study of learning domainsensitive word representations Bollegala et al, 2015Bollegala et al, , 2014. They basically learn separate embeddings of the same word for different domains.…”

Section: Introductionmentioning

confidence: 93%

Learning Domain-Sensitive and Sentiment-Aware Word Embeddings

Shi¹,

Fu²,

Lam³

2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Word embeddings have been widely used in sentiment classification because of their efficacy for semantic representations of words. Given reviews from different domains, some existing methods for word embeddings exploit sentiment information, but they cannot produce domainsensitive embeddings. On the other hand, some other existing methods can generate domain-sensitive word embeddings, but they cannot distinguish words with similar contexts but opposite sentiment polarity. We propose a new method for learning domain-sensitive and sentimentaware embeddings that simultaneously capture the information of sentiment semantics and domain sensitivity of individual words. Our method can automatically determine and produce domain-common embeddings and domain-specific embeddings. The differentiation of domaincommon and domain-specific words enables the advantage of data augmentation of common semantics from multiple domains and capture the varied semantics of specific words from different domains at the same time. Experimental results show that our model provides an effective way to learn domain-sensitive and sentimentaware word embeddings which benefit sentiment classification at both sentence level and lexicon term level.

show abstract

“…However, as described in Section 1, the meaning of a word vary from one domain to another, and must be considered. To the best of our knowledge, the only prior work studying the problem of word representation variation across domains is due to Bollegala et al (2014). Given a source and a target domain, they first select a set of pivots using pointwise mutual information, and create two distributional representa-tions for each pivot using their co-occurrence contexts in a particular domain.…”

Section: Related Workmentioning

confidence: 99%

“…For exam-ple, the phrase lightweight is often used in a positive sentiment in the portable electronics domain because a lightweight device is easier to carry around, which is a positive attribute for a portable electronic device. However, the same phrase has a negative sentiment assocition in the movie domain because movies that do not invoke deep thoughts in viewers are considered to be lightweight (Bollegala et al, 2014). However, existing word representation learning methods are agnostic to such domain-specific semantic variations of words, and capture semantics of words only within a single domain.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Cross-Domain Word Representation Learning

Bollegala¹,

Maehara²,

Kawarabayashi³

2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

Self Cite

View full text Add to dashboard Cite

Meaning of a word varies from one domain to another. Despite this important domain dependence in word semantics, existing word representation learning methods are bound to a single domain. Given a pair of source-target domains, we propose an unsupervised method for learning domain-specific word representations that accurately capture the domainspecific aspects of word semantics. First, we select a subset of frequent words that occur in both domains as pivots. Next, we optimize an objective function that enforces two constraints: (a) for both source and target domain documents, pivots that appear in a document must accurately predict the co-occurring non-pivots, and (b) word representations learnt for pivots must be similar in the two domains. Moreover, we propose a method to perform domain adaptation using the learnt word representations. Our proposed method significantly outperforms competitive baselines including the state-of-theart domain-insensitive word representations, and reports best sentiment classification accuracies for all domain-pairs in a benchmark dataset.

show abstract

Learning to Predict Distributions of Words Across Domains

Cited by 15 publications

References 23 publications

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

Learning Domain-Sensitive and Sentiment-Aware Word Embeddings

Unsupervised Cross-Domain Word Representation Learning

Contact Info

Product

Resources

About