Leveraging the Information in In-Domain Datasets for Transformer-Based Intent Detection

Büyük, Osman; Erden, Mustafa; Arslan, Levent M.

doi:10.1109/asyu52992.2021.9599055

Cited by 2 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experimental setup and results are provided in this section. In the experiments, we use BERTurk in https://huggingface.co/dbmdz/bert-base-turkish-cased as the baseline pre-trained transformer model since it provided the best accuracies in our previous study (Büyük et al, 2021). The model has 12 transformer layers with 12 attention heads.…”

Section: Experimental Setup and Resultsmentioning

confidence: 99%

“…In (Dündar et al, 2020), it is concluded that contextual word embeddings from transformers improves the intent detection accuracy compared to the classical machine learning models. In our previous study, we implement an intent detection system using the pre-trained transformer models (Büyük et al, 2021). In the study, we used three different intent detection datasets.…”

Section: Introductionmentioning

confidence: 99%

“…As a result, we assign higher weights to the classes which are less represented in the dataset. Using the weighted loss function together with the transfer learning procedure in (Büyük et al, 2021) significantly improves the intent detection accuracy. We achieve approximately 2% absolute accuracy improvement in percent detection rate over a baseline model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mitigating Data Imbalance Problem in Transformer-Based Intent Detection

Büyük¹,

Erden²,

Arslan³

2022

European Journal of Science and Technology

Self Cite

View full text Add to dashboard Cite

There are two major problems when deploying a practical intent detection system for a new customer. First, domain-specific data from the customer could be limited and imbalanced. Additionally, despite different customers might share the same domain, their intent categories might be different from each other. Thus, it might be difficult to combine the datasets collected for different customers into a single and larger one. In this paper, we use class weights in the loss computation to alleviate the data imbalance problem. The class weights are defined inversely proportional to the frequency of the class in the training set in order to give more influence to less observed classes. We also employ a two-pass fine-tuning procedure to utilize the information in different in-domain datasets. Experimental results show that intent detection performance is improved significantly when the weighted loss function is used together with the two-pass transfer learning procedure. The absolute performance improvement in percent detection accuracy is approximately 2% over a transformer-based baseline.

show abstract

Section: Experimental Setup and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mitigating Data Imbalance Problem in Transformer-Based Intent Detection

Büyük¹,

Erden²,

Arslan³

2022

European Journal of Science and Technology

Self Cite

View full text Add to dashboard Cite

show abstract

Computational Linguistics with Deep-Learning-Based Intent Detection for Natural Language Understanding

et al. 2022

View full text Add to dashboard Cite

Computational linguistics explores how human language is interpreted automatically and then processed. Research in this area takes the logical and mathematical features of natural language and advances methods and statistical procedures for automated language processing. Slot filling and intent detection are significant modules in task-based dialogue systems. Intent detection is a critical task in any natural language understanding (NLU) system and constitutes the base of a task-based dialogue system. In order to build high-quality, real-time conversational solutions for edge gadgets, there is a demand for deploying intent-detection methods on devices. This mandates an accurate, lightweight, and fast method that effectively operates in a resource-limited environment. Earlier works have explored the usage of several machine-learning (ML) techniques for detecting intent in user queries. In this article, we propose Computational Linguistics with Deep-Learning-Based Intent Detection and Classification (CL-DLBIDC) for natural language understanding. The presented CL-DLBIDC technique receives word embedding as input and learned meaningful features to determine the probable intention of the user query. In addition, the presented CL-DLBIDC technique uses the GloVe approach. In addition, the CL-DLBIDC technique makes use of the deep learning modified neural network (DLMNN) model for intent detection and classification. For the hyperparameter tuning process, the mayfly optimization (MFO) algorithm was used in this study. The experimental analysis of the CL-DLBIDC method took place under a set of simulations, and the results were scrutinized for distinct aspects. The simulation outcomes demonstrate the significant performance of the CL-DLBIDC algorithm over other DL models.

show abstract

Leveraging the Information in In-Domain Datasets for Transformer-Based Intent Detection

Cited by 2 publications

References 6 publications

Mitigating Data Imbalance Problem in Transformer-Based Intent Detection

Mitigating Data Imbalance Problem in Transformer-Based Intent Detection

Computational Linguistics with Deep-Learning-Based Intent Detection for Natural Language Understanding

Contact Info

Product

Resources

About