Spoken language understanding using long short-term memory neural networks

Yao, Kaisheng; Peng, Baolin; Zhang, Yu; Yu, Dong; Zweig, Geoffrey; Shi, Yangyang

doi:10.1109/slt.2014.7078572

Cited by 298 publications

(199 citation statements)

References 25 publications

Supporting

Mentioning

198

Contrasting

Order By: Relevance

“…RNN also produces a sequence of locally normalized output distributions, one for each word position. Thus, it can suffer from a label bias problem [6]. LSTM RNN has been shown to perform more effectively than the standard RNN at finding and exploiting longrange dependencies in the data [6].…”

Section: Rnn and Lstm Rnn Modelsmentioning

confidence: 99%

“…Thus, it can suffer from a label bias problem [6]. LSTM RNN has been shown to perform more effectively than the standard RNN at finding and exploiting longrange dependencies in the data [6]. One difference from the standard RNN is that the LSTM uses a memory cell with gate units, which are linear activation functions.…”

Section: Rnn and Lstm Rnn Modelsmentioning

confidence: 99%

“…RNNs that use long short-term memory (LSTM) cells have proven effective in handwriting recognition, language modeling, speech recognition, and language comprehension tasks [4]- [6].…”

Section: Introductionmentioning

confidence: 99%

“…LSTM RNN also produces a sequence of locally normalized output distributions, one for each word position as in a maximum entropy Markov model (MEMM). Thus, LSTM RNN can suffer from the same label bias problem as in MEMM [6].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LSTM-CRF Models for Named Entity Recognition

Lee

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYRecurrent neural networks (RNNs) are a powerful model for sequential data. RNNs that use long short-term memory (LSTM) cells have proven effective in handwriting recognition, language modeling, speech recognition, and language comprehension tasks. In this study, we propose LSTM conditional random fields (LSTM-CRF); it is an LSTMbased RNN model that uses output-label dependencies with transition features and a CRF-like sequence-level objective function. We also propose variations to the LSTM-CRF model using a gate recurrent unit (GRU) and structurally constrained recurrent network (SCRN). Empirical results reveal that our proposed models attain state-of-the-art performance for named entity recognition.

show abstract

Section: Rnn and Lstm Rnn Modelsmentioning

confidence: 99%

Section: Rnn and Lstm Rnn Modelsmentioning

confidence: 99%

“…RNNs that use long short-term memory (LSTM) cells have proven effective in handwriting recognition, language modeling, speech recognition, and language comprehension tasks [4]- [6].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

LSTM-CRF Models for Named Entity Recognition

Lee

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Different researchers use slightly different LSTM variants (Graves, 2013;Yao et al, 2014;Jozefowicz et al, 2015). We implemented the version of LSTM described by the following set of equations:…”

Section: Lstmmentioning

confidence: 99%

Translation Quality Estimation using Recurrent Neural Network

Patel¹,

Sasikumar²

2016

Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers

View full text Add to dashboard Cite

This paper describes our submission to the shared task on word/phrase level Quality Estimation (QE) in the First Conference on Statistical Machine Translation (WMT16).The objective of the shared task was to predict if the given word/phrase is a correct/incorrect (OK/BAD) translation in the given sentence. In this paper, we propose a novel approach for word level Quality Estimation using Recurrent Neural Network Language Model (RNN-LM) architecture. RNN-LMs have been found very effective in different Natural Language Processing (NLP) applications. RNN-LM is mainly used for vector space language modeling for different NLP problems. For this task, we modify the architecture of RNN-LM. The modified system predicts a label (OK/BAD) in the slot rather than predicting the word. The input to the system is a word sequence, similar to the standard RNN-LM. The approach is language independent and requires only the translated text for QE. To estimate the phrase level quality, we use the output of the word level QE system.

show abstract