Deep belief network based semantic taggers for spoken language understanding

Deoras, Anoop; Sarikaya, Ruhi

doi:10.21437/interspeech.2013-623

Cited by 44 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Addressed issue Approach [Yu et al 2011] Long-range state dependency Deep learning, CRF [Deng et al 2012] To extend DCN Kernel learning, deep learning, DCN, log-linear model [Deoras and Sarikaya 2013] Data sparsity problem (CRF)…”

Section: Papermentioning

confidence: 99%

“…[ Deoras and Sarikaya 2013] applied deep belief networks (DBN) for semantic tagging integrated with lexical, named entity, dependency parser based syntactic features and part of speech (POS) tags. A DBN is a stack of Restricted Boltzmann Machines (RBMs), where the input of one layer is the output of the previous one and each layer applies a sigmoid activation function on their inputs.…”

Section: Low Resource Data Setsmentioning

confidence: 99%

“…In most papers slot F1 is reported as the span based microaveraged F1 over all classes excluding O. The conlleval.py 2 script is regularly used ( [Daha and Hewavitharana 2019;Deoras and Sarikaya 2013;Liu et al 2019b]) to calculate F1 score, precision and recall with micro-averaging. 7.3.2 Token-based slot precision, recall and F1.…”

Section: Slot Labelling Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks have been deemed to proceed independently. However, more recently, joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks. This article is a compilation of past work in natural language understanding, especially joint intent classification and slot filling. We observe three milestones in this research so far: Intent detection to identify the speaker's intention, slot filling to label each word token in the speech/text, and finally, joint intent classification and slot filling tasks. In this article, we describe trends, approaches, issues, data sets, evaluation metrics in intent classification and slot filling. We also discuss representative performance values, describe shared tasks, and provide pointers to future work, as given in prior works. To interpret the state-of-the-art trends, we provide multiple tables that describe and summarise past research along different dimensions, including the types of features, base approaches, and dataset domain used.

show abstract

Section: Papermentioning

confidence: 99%

Section: Low Resource Data Setsmentioning

confidence: 99%

Section: Slot Labelling Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Another line of popular approaches is to train machine learning models on labeled training data, such as support vector machine (SVM) and Adaboost [9,29] . Approaches based on deep neural network technology have shown excellent performance, such as Deep belief networks (DBNs) and RNNs [25,5]. Slot filling can be treated as a sequence labeling task.…”

Section: Spoken Language Understandingmentioning

confidence: 99%

Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks

Wei¹,

Zeng²,

Liao³

2021

Preprint

View full text Add to dashboard Cite

Intent detection and slot filling are two fundamental tasks for building a spoken language understanding (SLU) system. Multiple deep learning-based joint models have demonstrated excellent results on the two tasks. In this paper, we propose a new joint model with a wheel-graph attention network (Wheel-GAT) which is able to model interrelated connections directly for intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges. Intent nodes can provide utterance-level semantic information for slot filling, while slot nodes can also provide local keyword information for intent. Experiments show that our model outperforms multiple baselines on two public datasets. Besides, we also demonstrate that using Bidirectional Encoder Representation from Transformer (BERT) model further boosts the performance in the SLU task.

show abstract

“…The input is the sentence consisting of a sequence of words, and the output is a sequence of slot/concept IDs, one for each word. [17] and [15] used deep belief networks (DBNs), and achieved superior results compared to CRF baselines. [51; 115; 66; 113] applied RNN for slot filling.…”

Section: Language Understandingmentioning

confidence: 99%

A Survey on Dialogue Systems: Recent Advances and New Frontiers

Chen,

Liu,

Yin

et al. 2017

Preprint

View full text Add to dashboard Cite

Dialogue systems have attracted more and more attention. Recent advances on dialogue systems are overwhelmingly contributed by deep learning techniques, which have been employed to enhance a wide range of big data applications such as computer vision, natural language processing, and recommender systems. For dialogue systems, deep learning can leverage a massive amount of data to learn meaningful feature representations and response generation strategies, while requiring a minimum amount of hand-crafting. In this article, we give an overview to these recent advances on dialogue systems from various perspectives and discuss some possible research directions. In particular, we generally divide existing dialogue systems into task-oriented and nontask-oriented models, then detail how deep learning techniques help them with representative algorithms and finally discuss some appealing research directions that can bring the dialogue system research into a new frontier.

show abstract

Deep belief network based semantic taggers for spoken language understanding

Cited by 44 publications

References 18 publications

A survey of joint intent detection and slot-filling models in natural language understanding

A survey of joint intent detection and slot-filling models in natural language understanding

Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks

A Survey on Dialogue Systems: Recent Advances and New Frontiers

Contact Info

Product

Resources

About