Intent Detection Problem Solving via Automatic DNN Hyperparameter Optimization

Abstract

Accurate intent detection-based chatbots are usually trained on larger datasets that are not available for some languages. Seeking the most accurate models, three English benchmark datasets that were human-translated into four morphologically complex languages (i.e., Estonian, Latvian, Lithuanian, Russian) were used. Two types of word embeddings (fastText and BERT), three types of deep neural network (DNN) classifiers (convolutional neural network (CNN); long short-term memory method (LSTM), and bidirectional LSTM (BiLSTM)), different DNN architectures (shallower and deeper), and various DNN hyperparameter values were investigated. DNN architecture and hyperparameter values were optimized automatically using the Bayesian method and random search. On three datasets of 2/5/8 intents for English, Estonian, Latvian, Lithuanian, and Russian languages, accuracies of 0.991/0.890/0.712, 0.972/0.890/0.644, 1.000/0.890/0.644, 0.981/0.872/0.712, and 0.972/0.881/0.661 were achieved, respectively. The BERT multilingual vectorization with the CNN classifier was proven to be a good choice for all datasets for all languages. Moreover, in the majority of models, the same set of optimal hyperparameter values was determined. The results obtained in this research were also compared with the previously reported values (where hyperparameter values of DNN models were selected by an expert). This comparison revealed that automatically optimized models are competitive or even more accurate when created with larger training datasets.

Correlation model of deflection, vehicle load, and temperature for in‐service bridge using deep learning and structural health monitoring

“…The optimization method of these hyperparameters is a widely existing and hot‐spot problem. At present, the more widely used hyperparametric optimization methods include grid search, random search, and Bayesian optimization 42–44 . In this study, a simple and effective grid search algorithm 42 is used to optimize the hyperparameters of GRU and LSTM neural network.…”

“…At present, the more widely used hyperparametric optimization methods include grid search, random search, and Bayesian optimization. [42][43][44] In this study, a simple and effective grid search algorithm 42 is used to optimize the hyperparameters of GRU and LSTM neural network. Using the network structures shown in Figure 2, the number of training iterations is set to 500.…”

Data quality evaluation for bridge structural health monitoring based on deep learning and frequency-domain information

“…At present, the commonly used hyperparametric optimization methods include grid search, random search, and Bayesian optimization. [50][51][52] In this study, the grid search algorithm 50 was used to optimize the hyper-parameters of CNN. The optimized learning rate was 0.003, and the learning rate decreased by epochs.…”

Chatbots for HIV Prevention and Care: a Narrative Review

“…Rule-based chatbots use a state machine and pre-defined question-and-answer structures to control conversations. On the other hand, AI-based chatbots utilize AI techniques for natural language understanding (NLU) and natural language generation (NLG), allowing them to understand natural language, maintain different conversation contexts, and generate fluid and coherent responses [ 2 , 5 ]. Building on the existing body of knowledge, this review uniquely synthesizes recent developments, advancements, and challenges in the application of chatbots and large language models (LLMs) in HIV prevention and care, along with providing an insightful perspective on future directions, thereby offering a comprehensive understanding of this rapidly evolving intersection of technology and healthcare.…”