The Impact of Data Challenges on Intent Detection and Slot Filling for the Home Assistant Scenario

Stoica, Anda; Kadar, Tibor; Lemnaru, Camelia; Potolea, Rodica; Dînşoreanu, Mihaela

doi:10.1109/iccp48234.2019.8959642

Cited by 8 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The drop in performance from the baseline to Scenario 1 is the largest and the qualitative analysis suggests that the model tends to memorize specific words rather than learn the meaning of the sentence. We believe this to be rooted in the fact that we use word embeddings to represent words, which are good at expressing semantic similarity, but fail to capture certain relationships such as synonymy and antonymy (as shown in [ 3 ]). In order to mitigate this issue, an improvement could be to intervene in the word vector model and adjust the embeddings.…”

Section: Discussionmentioning

confidence: 99%

“…The home assistant scenario was initially introduced in [ 3 ], and consists of multiple datasets that include several data and learning challenges that are likely to appear in such an application domain, considering the Romanian language. To the best of our knowledge, this is the first and only dataset for intent detection and slot filling for the Romanian language to date.…”

Section: Description Of the Home Assistant Scenariomentioning

confidence: 99%

“…We carry out a more detailed error analysis using the best performing model (SCN-100). We notice that many of the error types that were present in the Rasa NLU model [ 3 ] are also produced by the capsule network.…”

Section: Empirical Evaluationsmentioning

confidence: 99%

“…In this work, we use the words “utterance” and “sentence” interchangeably, but both are used to refer to a sentence in text format. We initially approached the home assistant scenario in [ 3 ], by generating several datasets which considered a set of language- and learning-specific challenges that are likely to occur in such a scenario: language variability (via synonymy, or other mechanisms, some specific to the Romanian language), missing values (for slots) and classification imbalance (for the various intent classes). A significant difficulty that resulted from the analysis performed in [ 3 ] was related to the misclassification of opposing intents.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Stoica

Kadar

Lemnaru

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

As virtual home assistants are becoming more popular, there is an emerging need for supporting languages other than English. While more wide-spread or popular languages such as Spanish, French or Hindi are already integrated into existing home assistants like Google Home or Alexa, integration of other less-known languages such as Romanian is still missing. This paper explores the problem of Natural Language Understanding (NLU) applied to a Romanian home assistant. We propose a customized capsule neural network architecture that performs intent detection and slot filling in a joint manner and we evaluate how well it handles utterances containing various levels of complexity. The capsule network model shows a significant improvement in intent detection when compared to models built using the well-known Rasa NLU tool. Through error analysis, we observe clear error patterns that occur systematically. Variability in language when expressing one intent proves to be the biggest challenge encountered by the model.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Description Of the Home Assistant Scenariomentioning

confidence: 99%

Section: Empirical Evaluationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Stoica

Kadar

Lemnaru

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [1], a home assistant scenario for the Romanian language was proposed and implemented. Corresponding datasets were generated and labeled, in order to train machine learning models for intent detection and slot filling.…”

Section: Introductionmentioning

confidence: 99%

Semantically Enriching Embeddings of Highly Inflectable Verbs for Improving Intent Detection in a Romanian Home Assistant Scenario

Rad

Muntean

Stoica

et al. 2021

Advances in Intelligent Data Analysis XIX

Self Cite

View full text Add to dashboard Cite

Word embeddings are known to encapsulate semantic similarity and have become the preferred representation solution for NLP models. However, they fail to identify the type of semantic relationship, which -in some applications -might be crucial. This paper adapts an existing solution for enhancing word embedding representations such as to better separate between synonyms and antonyms in an intent detection task applied to a Romanian home assistant scenario. Accounting for the morphological richness of the Romanian language, our method proposes an additional augmentation step, in order to generate conjugated pairs of antonym and synonym verbs. The generated pairs are run through the counterfitting step (inspired from literature), for which we propose a justified improvement for one of the hyperparameters. The evaluations performed on the home assistant scenario have shown that the pre-processing step has an essential role in reducing opposing intent errors in the classification model (by almost two thirds).

show abstract

Evaluation of Embeddings in Medication Domain for Spanish Language Using Joint Natural Language Understanding

Roca

Rosset

García

et al. 2020

8th European Medical and Biological Engineering Conference

View full text Add to dashboard Cite

Word embeddings have been widely used in Natural Language Processing as the input to neural networks. Such word embeddings can help in the understanding of the final objective and the keywords in a sentence. As such, in this work, we study the impact of different word embeddings trained with general and specific corpora using Joint Natural Language Understanding in a Spanish medication domain. We generate data using templates for training the model. The model is used for intent detection and slot-filling. We compare word2vec and fastText as word embeddings and ELMo and BERT as language models. We use three different corpora to train the embeddings: the training data generated for this scenario, the Spanish Wikipedia as general domain and the Spanish drug database as specialized data. The best result was obtained with word2vec continuous bag of words model learned with Spanish Wikipedia, obtaining a 71.77% F1-score for intent detection, an intent accuracy of 69.37% and a 74.36% F1-score for slot-filling.

show abstract

The Impact of Data Challenges on Intent Detection and Slot Filling for the Home Assistant Scenario

Cited by 8 publications

References 11 publications

Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Semantically Enriching Embeddings of Highly Inflectable Verbs for Improving Intent Detection in a Romanian Home Assistant Scenario

Evaluation of Embeddings in Medication Domain for Spanish Language Using Joint Natural Language Understanding

Contact Info

Product

Resources

About