A Domain-Specific Generative Chatbot Trained from Little Data

Abstract

Accurate generative chatbots are usually trained on large datasets of question–answer pairs. Despite such datasets not existing for some languages, it does not reduce the need for companies to have chatbot technology in their websites. However, companies usually own small domain-specific datasets (at least in the form of an FAQ) about their products, services, or used technologies. In this research, we seek effective solutions to create generative seq2seq-based chatbots from very small data. Since experiments are carried out in English and morphologically complex Lithuanian languages, we have an opportunity to compare results for languages with very different characteristics. We experimentally explore three encoder–decoder LSTM-based approaches (simple LSTM, stacked LSTM, and BiLSTM), three word embedding types (one-hot encoding, fastText, and BERT embeddings), and five encoder–decoder architectures based on different encoder and decoder vectorization units. Furthermore, all offered approaches are applied to the pre-processed datasets with removed and separated punctuation. The experimental investigation revealed the advantages of the stacked LSTM and BiLSTM encoder architectures and BERT embedding vectorization (especially for the encoder). The best achieved BLUE on English/Lithuanian datasets with removed and separated punctuation was ~0.513/~0.505 and ~0.488/~0.439, respectively. Better results were achieved with the English language, because generating different inflection forms for the morphologically complex Lithuanian is a harder task. The BLUE scores fell into the range defining the quality of the generated answers as good or very good for both languages. This research was performed with very small datasets having little variety in covered topics, which makes this research not only more difficult, but also more interesting. Moreover, to our knowledge, it is the first attempt to train generative chatbots for a morphologically complex language.

Text Messaging-Based Medical Diagnosis Using Natural Language Processing and Fuzzy Logic

“…To evaluate the performance of the developed service, we used the Bilingual Evaluation Understudy (BLEU) score [71], which has become a typical metric for evaluating chatbot services [72,73]. BLEU scores an output response from the service as compared to the reference, where a BLEU score ranges from 0 to 1.…”

Online Textual Symptomatic Assessment Chatbot Based on Q&A Weighted Scoring for Female Breast Cancer Prescreening

“…Considering that data availability for research in the closed domain of healthcare sector are often either scarce [14,25,26] due to strict medical ethics enacted to protect the privacy of patients' information or that the available data are unusable [27] poses some concerns regarding the application of DNNs in this regard. Although authors such as Kapociūtė-Dzikienė [28] have conducted experiments on DNNbased chatbot (e.g., stacked bidirectional long short-term memory (BiLSTM)) and achieved interesting results after training with small-scale dataset for a closed-domain task, such approaches may not yet be fully accredited for implementation in critical service domains such as the healthcare sector, where dialog errors can result to fatal implications [12,14]. Meanwhile, contrary to DNNs, scholars [9,14,16,19] have evidently demonstrated the practical effectiveness of modelling medical chatbots with rule-based NLP frameworks that are based on traditional hand-crafted machine-learning approaches.…”

“…Further research directions relevant to our work borders on the implementation of text-based natural language understanding for dialog systems. On this subject, English and Chinese texts are the most researched languages [28]. A study by Zhang et al [29] highlighted the current social phenomenon of today's multilingualism of which the authors [29] proposed a two-stage Chinese-English mixed text normalization module to benefit NLP preprocessing tasks, owing to the recent prevalent norm of Chinese-speaking locals frequently mixing informal Chinese texts with English words, especially in social messaging platforms.…”

A Technical Survey on the Modeling of Topical Bot

“…Hybrid approaches utilizing retrieval in combination with generative models are genuinely new and have shown promising outcomes in recent years, typically with sequenceto-sequence approaches with some variants. [2] Dialog Management: This is responsible for making the conversation engaging and finding the results related to the conversation. Once response generation is done, the dialog manager works in selecting the right response and the context.…”