Deep Learning--based Text Classification

Minaee, Shervin; Kalchbrenner, Nal; Wang, Zhaoxia; Nikzad, Narjes; Chenaghlu, Meysam; Gao, Jianfeng

doi:10.1145/3439726

Cited by 882 publications

(369 citation statements)

References 128 publications

Supporting

Mentioning

337

Contrasting

Unclassified

Order By: Relevance

“…In [48], the authors propose an exhaustive review of the state of the art, where they describe multiple state-of-the-art deep learning models, currently in use for Natural Language Applications, as well multiple datasets for training and testing the proposed architectures. They highlight the use of CNN, LSTM, RNN, and FCNN.…”

Section: Discussionmentioning

confidence: 99%

Sentiment Analysis Techniques Applied to Raw-Text Data from a Csq-8 Questionnaire about Mindfulness in Times of COVID-19 to Improve Strategy Generation

Jojoa

Castillo-Sánchez

Garcia‐Zapirain

et al. 2021

IJERPH

View full text Add to dashboard Cite

The use of artificial intelligence in health care has grown quickly. In this sense, we present our work related to the application of Natural Language Processing techniques, as a tool to analyze the sentiment perception of users who answered two questions from the CSQ-8 questionnaires with raw Spanish free-text. Their responses are related to mindfulness, which is a novel technique used to control stress and anxiety caused by different factors in daily life. As such, we proposed an online course where this method was applied in order to improve the quality of life of health care professionals in COVID 19 pandemic times. We also carried out an evaluation of the satisfaction level of the participants involved, with a view to establishing strategies to improve future experiences. To automatically perform this task, we used Natural Language Processing (NLP) models such as swivel embedding, neural networks, and transfer learning, so as to classify the inputs into the following three categories: negative, neutral, and positive. Due to the limited amount of data available—86 registers for the first and 68 for the second—transfer learning techniques were required. The length of the text had no limit from the user’s standpoint, and our approach attained a maximum accuracy of 93.02% and 90.53%, respectively, based on ground truth labeled by three experts. Finally, we proposed a complementary analysis, using computer graphic text representation based on word frequency, to help researchers identify relevant information about the opinions with an objective approach to sentiment. The main conclusion drawn from this work is that the application of NLP techniques in small amounts of data using transfer learning is able to obtain enough accuracy in sentiment analysis and text classification stages.

show abstract

Section: Discussionmentioning

confidence: 99%

Sentiment Analysis Techniques Applied to Raw-Text Data from a Csq-8 Questionnaire about Mindfulness in Times of COVID-19 to Improve Strategy Generation

Jojoa

Castillo-Sánchez

Garcia‐Zapirain

et al. 2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…In [46] the authors propose an exhaustive review of the state of the art, where they describe multiple State of the Art deep learning models, currently in use for Natural Language Applications, as well multiple datasets for training and testing the proposed architectures. They highlight the use of CNN, LSTM, RNN and FCNN.…”

Section: Discussionmentioning

confidence: 99%

Sentiment Analysis Techniques Applied to Raw-Text Data from a CSQ-8 Questionnaire About Mindfulness in Times of Covid-19 to Improve Strategy Generation

Jojoa¹,

Castillo-Sánchez²,

Garcia‐Zapirain³

et al. 2021

Preprint

View full text Add to dashboard Cite

The aim of this study was to build a tool to analyze, using artificial intelligence, the sentiment perception of users who answered two questions from the CSQ – 8 questionnaires with raw Spanish free-text. Their responses are related to mindfulness, which is a novel technique used to control stress and anxiety caused by different factors in daily life. As such, we proposed an online course where this method was applied in order to improve the quality of life of health care professionals in COVID 19 pandemic times. We also carried out an evaluation of the satis-faction level of the participants involved, with a view to establishing strategies to improve fu-ture experiences. To automatically perform this task, we used Natural Language Processing (NLP) models such as swivel embedding, neural networks and transfer learning, so as to classify the inputs into the following 3 categories: negative, neutral and positive. Due to the lim-ited amount of data available - 86 registers for the first and 68 for the second - transfer learning techniques were required. The length of the text had no limit from the user’s standpoint, and our approach attained a maximum accuracy of 93.02 % and 90.53 % respectively based on ground truth labeled by 3 experts. Finally, we proposed a complementary analysis, using com-puter graphic text representation based on word frequency, to help researchers identify relevant information about the opinions with an objective approach to sentiment. The main conclusion drawn from this work is that the application of NLP techniques in small amounts of data using transfer learning is able to obtain enough accuracy in sentiment analysis and text classification stages

show abstract

“…On the other hands, the second-order method determines its next search direction from Hessian matrix of the loss function as shown in Equation (9). Accordingly, the secondorder method can perform better solution search in the complicated optimization terrains with a lot of saddle points or local minimums [29].…”

Section: Overview Of Optimization Methods For Machine Learningmentioning

confidence: 99%

“…In particular, modern neural network models are consisted of deeper layers and more weights than traditional ones to maximize their performance. Accordingly, the latest deep learning models have shown notable abilities in many real-world applications, for example, computer visions (CV) [1,2], data analysis [3,4], personalized services [5,6], internet of things (IoT) [7,8], and natural language processing (NLP) [9,10], et al Among them, particularly, the CV task involving image classification and image semantic segmentation is one of the applications in which the deep learning models have been most actively used. Accordingly, many studies to improve the image processing ability of CNNs are being actively conducted.…”

Section: Introductionmentioning

confidence: 99%

HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks

Kim

Choi

2021

Sensors

View full text Add to dashboard Cite

As the performance of devices that conduct large-scale computations has been rapidly improved, various deep learning models have been successfully utilized in various applications. Particularly, convolution neural networks (CNN) have shown remarkable performance in image processing tasks such as image classification and segmentation. Accordingly, more stable and robust optimization methods are required to effectively train them. However, the traditional optimizers used in deep learning still have unsatisfactory training performance for the models with many layers and weights. Accordingly, in this paper, we propose a new Adam-based hybrid optimization method called HyAdamC for training CNNs effectively. HyAdamC uses three new velocity control functions to adjust its search strength carefully in term of initial, short, and long-term velocities. Moreover, HyAdamC utilizes an adaptive coefficient computation method to prevent that a search direction determined by the first momentum is distorted by any outlier gradients. Then, these are combined into one hybrid method. In our experiments, HyAdamC showed not only notable test accuracies but also significantly stable and robust optimization abilities when training various CNN models. Furthermore, we also found that HyAdamC could be applied into not only image classification and image segmentation tasks.

show abstract

Deep Learning--based Text Classification

Cited by 882 publications

References 128 publications

Sentiment Analysis Techniques Applied to Raw-Text Data from a Csq-8 Questionnaire about Mindfulness in Times of COVID-19 to Improve Strategy Generation

Sentiment Analysis Techniques Applied to Raw-Text Data from a Csq-8 Questionnaire about Mindfulness in Times of COVID-19 to Improve Strategy Generation

Sentiment Analysis Techniques Applied to Raw-Text Data from a CSQ-8 Questionnaire About Mindfulness in Times of Covid-19 to Improve Strategy Generation

HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks

Contact Info

Product

Resources

About