Recurrent conditional random field for language understanding

Yao, Kaisheng; Peng, Baolin; Zweig, Geoffrey; Yu, Dong; Li, Xiaolong; Gao, Feng

doi:10.1109/icassp.2014.6854368

Cited by 109 publications

(78 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Peng et al (2009) add a non-linear neural network layer to a linear-chain CRF and Do and Artires (2010) apply a similar approach to more general Markov network structures. Yao et al (2014) and Zheng et al (2015) introduce recurrence into the model and finally combine CRFs and LSTMs. These neural CRF models are limited to sequence labeling tasks where exact inference is possible, while our model works well when exact inference is intractable.…”

Section: Related Neural Crf Workmentioning

confidence: 99%

Globally Normalized Transition-Based Neural Networks

Andor¹,

Alberti²,

Weiss³

et al. 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

378

429

View full text Add to dashboard Cite

We introduce a globally normalized transition-based neural network model that achieves state-of-the-art part-ofspeech tagging, dependency parsing and sentence compression results. Our model is a simple feed-forward neural network that operates on a task-specific transition system, yet achieves comparable or better accuracies than recurrent models. We discuss the importance of global as opposed to local normalization: a key insight is that the label bias problem implies that globally normalized models can be strictly more expressive than locally normalized models.

show abstract

Section: Related Neural Crf Workmentioning

confidence: 99%

Globally Normalized Transition-Based Neural Networks

Andor¹,

Alberti²,

Weiss³

et al. 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

378

429

View full text Add to dashboard Cite

show abstract

“…The other direction is to optimize the sequence-level discrimination criterion. For example, recurrent conditional random fields (RCRFs) (Yao et al, 2014b) is trained to optimize the sequence conditional likelihood rather than minimizing frame level cross-entropy applied in conventional RNN based sequence labeling (Prez-ortiz et al, 2001;Yao et al, 2013;Mikolov et al, 2010;Shi et al, 2015;Mesnil et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Recurrent Support Vector Machines For Slot Tagging In Spoken Language Understanding

Shi¹,

Yao²,

Hu³

et al. 2016

Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

We propose recurrent support vector machine (RSVM) for slot tagging. This model is a combination of the recurrent neural network (RNN) and the structured support vector machine. RNN extracts features from the input sequence. The structured support vector machine uses a sequence-level discriminative objective function. The proposed model therefore combines the sequence representation capability of an RNN with the sequence-level discriminative objective. We have observed new state-ofthe-art results on two benchmark datasets and one private dataset. RSVM obtained statistical significant 4% and 2% relative average F1 score improvement on ATIS dataset and Chunking dataset, respectively. Out of eight domains in Cortana live log dataset, RSVM achieved F1 score improvement on seven domains. Experiments also show that RSVM significantly speeds up the model training by skipping the weight updating for non-support vector training samples, compared against training using RNN with CRF or minimum cross-entropy objectives.

show abstract

“…Our approach is an advance over previous research on recurrent conditional random fields (R-CRF) for language comprehension [10]. R-CRF models use RNN to exploit long-range dependencies in the sequence of words.…”

Section: Introductionmentioning

confidence: 99%

LSTM-CRF Models for Named Entity Recognition

Lee

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYRecurrent neural networks (RNNs) are a powerful model for sequential data. RNNs that use long short-term memory (LSTM) cells have proven effective in handwriting recognition, language modeling, speech recognition, and language comprehension tasks. In this study, we propose LSTM conditional random fields (LSTM-CRF); it is an LSTMbased RNN model that uses output-label dependencies with transition features and a CRF-like sequence-level objective function. We also propose variations to the LSTM-CRF model using a gate recurrent unit (GRU) and structurally constrained recurrent network (SCRN). Empirical results reveal that our proposed models attain state-of-the-art performance for named entity recognition.

show abstract

Recurrent conditional random field for language understanding

Cited by 109 publications

References 23 publications

Globally Normalized Transition-Based Neural Networks

Globally Normalized Transition-Based Neural Networks

Recurrent Support Vector Machines For Slot Tagging In Spoken Language Understanding

LSTM-CRF Models for Named Entity Recognition

Contact Info

Product

Resources

About