Chatbot Interaction with Artificial Intelligence: human data augmentation with T5 and language transformer ensemble for text classification

Bird, Jordan J.; Ekárt, Anikó; Faria, Diego R.

doi:10.1007/s12652-021-03439-8

Cited by 43 publications

(27 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BERT and its derivatives (RoBERTa, ALBERT, BART, etc.) are state-of-the-art models that often outperform deep learning and classical models due to their simple yet advanced technique [96], [105], [106]. The BERT model includes many parameters, operates on a large scale, and has high latency [107].…”

Section: Text Classificationmentioning

confidence: 99%

Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification

Chi,

Tang,

Salleh

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Natural language processing (NLP) is a research field that provides huge potential to automate accounting tasks dealing with text data. This research studies the application of NLP in automatically categorizing invoices based on the invoice text description. The study employs semantic enrichment, data augmentation, and deep learning to address the NLP unique issues posed by the inherent short text and multiclass imbalance nature of invoice descriptions. Semantic enrichment was done using labels as an information source. Training data was artificially increased with either WordNet synonym replacement, Global Vectors for Word Representation (GloVe) word replacement, or Bidirectional Encoder Representations from Transformers (BERT) word replacement method. Each training dataset was then supplied for training with one nondeep learning classifier and two deep learning classifiers respectively, namely Linear Support Vector Machine (LSVM), Bidirectional Long Short-Term Memory (Bi-LSTM), and BERT. Overall, the semantically enriched, WordNet augmented training set paired with the BERT classifier yielded the best results, successfully preserving semantics, reducing noise and overfitting while improving accuracy per class, achieving an increase of performance up to 20 ppts for macro F1 score and 6.7 percentage points (ppts) for accuracy.

show abstract

Section: Text Classificationmentioning

confidence: 99%

Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification

Chi,

Tang,

Salleh

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Researchers around the world are inspired to develop the state of-the-art chatbots by embedding different machine learning algorithms such as naïve Bayes algorithm and support vector machine (SVM) [4,5].…”

Section: Related Workmentioning

confidence: 99%

Machine Learning Chatbot for Sentiment Analysis of Covid-19 Tweets

Assayed¹,

Shaalan²,

Alkhatib³

et al. 2023

Computer Networks &Amp; Communications

View full text Add to dashboard Cite

The various types of social media were increased rapidly, as people’s need to share knowledge between others. In fact, there are various types of social media apps and platforms such as Facebook, Twitter, Reddit, Instagram, and others. Twitter remains one of the most popular social application that people use for sharing their emotional states. However, this has increased particularly during the COVID-19 pandemic. In this paper, we proposed a chatbot for evaluating the sentiment analysis by using machine learning algorithms. The authors used a dataset of tweets from Kaggle’s website, and that includes 41157 tweets that are related to the COVID-19. These tweets were classified and labelled to four categories: Extremely positive, positive, neutral, negative, and extremely negative. In this study, we applied Machine Learning algorithms, Support Vector Machines (SVM), and the Naïve Bayes (NB) algorithms and accordingly, we compared the accuracy between them. In addition to that, the classifiers were evaluated and compared after changing the test split ratio. The result shows that the accuracy performance of SVM algorithm is better than Naïve Bayes algorithm, even though Naïve Bayes perform poorly with low accuracy, but it trained the data faster comparing to SVM.

show abstract

“…This model has previously been successfully leveraged for data augmentation through paraphrase generation in a real-world application pertaining to the development of a human-robot interaction framework [106]. We also explored an alternative T5 model fine-tuned on the Google PAWS dataset for this task, but it was not utilised as most paraphrases generated were completely duplicate repetitions of the original questions.…”

Section: ) Utilising the T5 Model For Paraphrase Generationmentioning

confidence: 99%

Automated Question-Answering for Interactive Decision Support in Operations & Maintenance of Wind Turbines

Chatterjee

Dethlefs

2022

IEEE Access

View full text Add to dashboard Cite

Intelligent question-answering (QA) systems have witnessed increased interest in recent years, particularly in their ability to facilitate information access, data interpretation or decision support. The wind energy sector is one of the most promising sources of renewable energy, yet turbines regularly suffer from failures and operational inconsistencies, leading to downtimes and significant maintenance costs. Addressing these issues requires rapid interpretation of complex and dynamic data patterns under time-critical conditions. In this article, we present a novel approach that leverages interactive, natural language-based decision support for operations & maintenance (O&M) of wind turbines. The proposed interactive QA system allows engineers to pose domain-specific questions in natural language, and provides answers (in natural language) based on the automated retrieval of information on turbine subcomponents, their properties and interactions, from a bespoke domain-specific knowledge graph. As data for specific faults is often sparse, we propose the use of paraphrase generation as a way to augment the existing dataset. Our QA system leverages encoder-decoder models to generate Cypher queries to obtain domain-specific facts from the KG database in response to user-posed natural language questions. Experiments with an attention-based sequence-to-sequence (Seq2Seq) model and a transformer show that the transformer accurately predicts up to 89.75% of responses to input questions, outperforming the Seq2Seq model marginally by 0.76%, though being 9.46 times more computationally efficient. The proposed QA system can help support engineers and technicians during O&M to reduce turbine downtime and operational costs, thus improving the reliability of wind energy as a source of renewable energy.

show abstract

Chatbot Interaction with Artificial Intelligence: human data augmentation with T5 and language transformer ensemble for text classification

Cited by 43 publications

References 54 publications

Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification

Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification

Machine Learning Chatbot for Sentiment Analysis of Covid-19 Tweets

Automated Question-Answering for Interactive Decision Support in Operations & Maintenance of Wind Turbines

Contact Info

Product

Resources

About