A Comparison of Deep Learning Methods for Language Understanding

Korpusik, Mandy; Liu, Zoe; Glass, James

doi:10.21437/interspeech.2019-1262

Cited by 12 publications

(6 citation statements)

References 23 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ Korpusik et al 2019] perform a useful comparison of BiGRU, CNN and BERT based models with the then new BERT outperforming the other candidates.…”

Section: 21mentioning

confidence: 99%

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks have been deemed to proceed independently. However, more recently, joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks. This article is a compilation of past work in natural language understanding, especially joint intent classification and slot filling. We observe three milestones in this research so far: Intent detection to identify the speaker's intention, slot filling to label each word token in the speech/text, and finally, joint intent classification and slot filling tasks. In this article, we describe trends, approaches, issues, data sets, evaluation metrics in intent classification and slot filling. We also discuss representative performance values, describe shared tasks, and provide pointers to future work, as given in prior works. To interpret the state-of-the-art trends, we provide multiple tables that describe and summarise past research along different dimensions, including the types of features, base approaches, and dataset domain used.

show abstract

“…[ Korpusik et al 2019] perform a useful comparison of BiGRU, CNN and BERT based models with the then new BERT outperforming the other candidates.…”

Section: 21mentioning

confidence: 99%

A survey of joint intent detection and slot-filling models in natural language understanding

Weld¹,

Huang²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Over the past five years, the studies developed for SLU task are based on neural network architectures (Yao et al, 2014;Mesnil et al, 2015;Guo et al, 2014;Zhang and Wang, 2016;Dinarelli et al, 2017;Simonnet et al, 2017;Korpusik et al, 2019;Ghannay et al, 2020). Recent approaches take benefit from contextual or language model embeddings such as BERT (Devlin et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

“…Recent approaches take benefit from contextual or language model embeddings such as BERT (Devlin et al, 2019). Korpusik et al (2019) investigated the transfer ability of a pre-trained BERT representation for English SLU tasks. But, as far as we know, there are no such studies on a French SLU task.…”

Section: Introductionmentioning

confidence: 99%

Neural Networks approaches focused on French Spoken Language Understanding: application to the MEDIA Evaluation Task

Ghannay¹,

Servan²,

Rosset³

2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

In this paper, we present a study on a French Spoken Language Understanding (SLU) task: the MEDIA task. Many works and studies have been proposed for many tasks, but most of them are focused on English language and tasks. The exploration of a richer language like French within the framework of a SLU task implies to recent approaches to handle this difficulty. Since the MEDIA task seems to be one of the most difficult, according to several previous studies, we propose to explore Neural Networks approaches focusing of three aspects: firstly, the Neural Network inputs and more specifically the word embeddings; secondly, we compared French version of BERT against the best setup through different ways; Finally, the comparison against State-of-the-Art approaches. Results show that the word embeddings trained on a small corpus need to be updated during SLU model training. Furthermore, the French BERT fine-tuned approaches outperform the classical Neural Network Architectures and achieves state of the art results. However, the contextual embeddings extracted from one of the French BERT approaches achieve comparable results in comparison to word embedding, when integrated into the proposed neural architecture.

show abstract

“…Those representation algorithms can be classified into three categories: Vector Space Model (VSM), Theme Model (TM) and Neural Network Model (NNM). According to [1], in NNM, a representation based on simple neural network outperforms all of the VSM and TM based approaches. Especially, a novel self-attentive structure [2] in NNM is growing rapidly and shows the best capabilities in feature extraction through numerous real-world tasks [3][4][5].…”

Section: Introductionmentioning

confidence: 99%

“…Apparently, single embedding is impossible to vectorize all the features from the text. A common approach is to concatenate two embeddings, as it is shown by equation (1), where ℎ and ℎ are two types of embeddings. = ℎ ⊕ ℎ = {ℎ , ℎ , … , ℎ , ℎ , ℎ , … , ℎ }…”

Section: Introductionmentioning

confidence: 99%

A Novel Ensemble Representation Framework for Sentiment Classification

Sun

Hameed

Wang

2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Text representation has a critical impact on the accuracy of text classifiers which is imperative to be strengthened. On the other hand, the question of how the state-of-the-art embeddings outperform previous approaches cannot be well explained. To advance text representation and better understand the internal mechanism, we propose a novel end-to-end framework named Ensemble Framework for Text Embedding (EFTE), which weightedly combines diverse embeddings and simultaneously represents sentences' and tokens' features in a more reasonable way. According to the experimental results in sentiment classification, our proposed embedding apparently improves the effectiveness compared to six single embeddings. Moreover, the importance of each embedding in terms of EFTE integration and how different embeddings influence the results by classification are discussed.

show abstract

A Comparison of Deep Learning Methods for Language Understanding

Cited by 12 publications

References 23 publications

A survey of joint intent detection and slot-filling models in natural language understanding

A survey of joint intent detection and slot-filling models in natural language understanding

Neural Networks approaches focused on French Spoken Language Understanding: application to the MEDIA Evaluation Task

A Novel Ensemble Representation Framework for Sentiment Classification

Contact Info

Product

Resources

About