Sentence Modeling with Gated Recursive Neural Network

Chen, Xinchi; Qiu, Xipeng; Zhu, Chenxi; Wu, Shiyu; Huang, Xuanjing

doi:10.18653/v1/d15-1092

Cited by 32 publications

(26 citation statements)

References 13 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The most primitive embedding representation is the One-Hot encoding, which is simple but has two major shortages: (1) It supposes the words are independent from each other both semantically and grammatically. Hence, it cannot reflect the correlation between two words even they actually have; (2) As the lexicon grows, the dimensions of the vector increase drastically so that a ''dimensional disaster'' will burst, that will lead to a high cost for subsequent computations. To overcome these drawbacks, Hinton propose a distributed representation of word vectors [11], exploiting a fixed-length vector to represent words.…”

Section: Related Workmentioning

confidence: 99%

MPSC: A Multiple-Perspective Semantics-Crossover Model for Matching Sentences

Peng

Liu

2019

IEEE Access

View full text Add to dashboard Cite

Sentence matching is crucial to many natural language processing (NLP) tasks. Generally, the degree of matching is measured from either of the two perspectives: topic-based match or semantic-based match. The former is to investigate if two sentences discuss the same topic, and the latter performs a deep level semantic matching of texts, which is currently the highlight in research. Deep semantic matching requires adequate modeling from the internal structure of the language objects as well as their interactions. To achieve this goal, this paper proposes a multiple-perspective semantics-crossover (MPSC) model for modeling the semantic-based match of two sentences. The model extracts the matching information of two sentences from the semantic interaction information generated from different angles, so as to calculate the matching degree of the two sentences. The MPSC model not only captures rich matching patterns at different levels but also acquires interactive features from different semantic angles. It can be used to address some important issues in NLP fields, such as information matching in text retrieval, question-answer matching in the Q&A system, and so on. The experimental results show that our proposed model of MPSC has better effectiveness than some popular semantic matching approaches.INDEX TERMS Natural language processing, neural networks, semantics matching, text analysis.

show abstract

Section: Related Workmentioning

confidence: 99%

MPSC: A Multiple-Perspective Semantics-Crossover Model for Matching Sentences

Peng

Liu

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…PV slightly surpasses MTLE on IMDB (91.7 against 91.3), as sentences from IMDB are much longer than SST and MDSD, which require stronger capabilities of long-term dependency learning. In this paper, we mainly focus the idea and effects of integrating label embedding with multi-task learning, so we just apply (Graves 2013) to realize Le I and Le L , which can be further implemented by other more effective sentence learning models (Liu et al 2015a;Chen et al 2015) and produce better performances.…”

Section: Comparisons With State-of-the-art Modelsmentioning

confidence: 99%

Multi-Task Label Embedding for Text Classification

Zhang¹,

Xiao²,

Chen³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Multi-task learning in text classification leverages implicit correlations among related tasks to extract common features and yield performance gains. However, most previous works treat labels of each task as independent and meaningless onehot vectors, which cause a loss of potential information and makes it difficult for these models to jointly learn three or more tasks. In this paper, we propose Multi-Task Label Embedding to convert labels in text classification into semantic vectors, thereby turning the original tasks into vector matching tasks. We implement unsupervised, supervised and semisupervised models of Multi-Task Label Embedding, all utilizing semantic correlations among tasks and making it particularly convenient to scale and transfer as more tasks are involved. Extensive experiments on five benchmark datasets for text classification show that our models can effectively improve performances of related tasks with semantic representations of labels and additional information from each other.

show abstract

“…RTfpass Chen et al, 2015] and general-purpose sentence representations [Le and Mikolov, 2014;Kiros et al, 2015;Kenter et al, 2016;Wieting et al, 2016]. Our work addresses the problem of learning general-purpose sentence representations that capture textual semantics and perform robustly across tasks.…”

Section: Modelsmentioning

confidence: 99%

Learning Sentence Representation with Guidance of Human Attention

Wang

Zhang

Zong

2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Recently, much progress has been made in learning general-purpose sentence representations that can be used across domains. However, most of the existing models typically treat each word in a sentence equally. In contrast, extensive studies have proven that human read sentences efficiently by making a sequence of fixation and saccades. This motivates us to improve sentence representations by assigning different weights to the vectors of the component words, which can be treated as an attention mechanism on single sentences. To that end, we propose two novel attention models, in which the attention weights are derived using significant predictors of human reading time, i.e., Surprisal, POS tags and CCG supertags. The extensive experiments demonstrate that the proposed methods significantly improve upon the state-of-the-art sentence representation models.

show abstract

Sentence Modeling with Gated Recursive Neural Network

Cited by 32 publications

References 13 publications

MPSC: A Multiple-Perspective Semantics-Crossover Model for Matching Sentences

MPSC: A Multiple-Perspective Semantics-Crossover Model for Matching Sentences

Multi-Task Label Embedding for Text Classification

Learning Sentence Representation with Guidance of Human Attention

Contact Info

Product

Resources

About