Traditional Machine Learning Models and Bidirectional Encoder Representations From Transformer (BERT)–Based Automatic Classification of Tweets About Eating Disorders: Algorithm Development and Validation Study

Abstract

Background Eating disorders affect an increasing number of people. Social networks provide information that can help. Objective We aimed to find machine learning models capable of efficiently categorizing tweets about eating disorders domain. Methods We collected tweets related to eating disorders, for 3 consecutive months. After preprocessing, a subset of 2000 tweets was labeled: (1) messages written by people suffering from eating disorders or not, (2) messages promoting suffering from eating disorders or not, (3) informative messages or not, and (4) scientific or nonscientific messages. Traditional machine learning and deep learning models were used to classify tweets. We evaluated accuracy, F1 score, and computational time for each model. Results A total of 1,058,957 tweets related to eating disorders were collected. were obtained in the 4 categorizations, with The bidirectional encoder representations from transformer–based models had the best score among the machine learning and deep learning techniques applied to the 4 categorization tasks (F1 scores 71.1%-86.4%). Conclusions Bidirectional encoder representations from transformer–based models have better performance, although their computational cost is significantly higher than those of traditional techniques, in classifying eating disorder–related tweets.

An Analysis of French-Language Tweets About COVID-19 Vaccines: Supervised Learning Approach

“…The size of the data set is quite similar to those of Kummervold et al [ 17 ] (1633 tweets for training and 544 for testing) and Benítez-Andrades et al [ 26 ] (n=1400 for training and n=600 for testing). Furthermore, the benefit of using a pretrained model such as the CamemBERT is that a large data set is not required to obtain good results.…”

“…This accuracy is slightly higher than that obtained by BERT for the same topic (vaccines) [ 17 ] and in the same range as previous findings [ 16 , 29 ]. However, CamemBERT obtained a better accuracy (78.7%-87.8%) in a study using dichotomous labels for tweets about eating disorders and using a preprocessing step, reducing the initial number of tweets by 2 [ 26 ]. However, by limiting the analysis to long tweets (170 or more characters, in accordance with the statistical analysis conducted on the performance of the model), the accuracy of classification model 2 improved significantly (from 62.9% to 72.4% for the F1-score).…”

The research foundation for COVID-19 vaccine development

“…We note that prior to using BERT, we have attempted K-means clustering technique that failed at clustering the documents into interpretable topics. Also, in previous studies (Benitez-Andrades et al, 2022;Bilal and Almazroi, 2022) the authors observed that BERT-based classifiers outperform bag-of-words approaches. Though BERT-based models can be computationally expensive (Bhattacharjee et al, 2020), we utilized BERTopic to have a better accuracy in classifying documents into interpretable topics.…”

Year 2022 in Medical Natural Language Processing: Availability of Language Models as a Step in the Democratization of NLP in the Biomedical Area

“…• when other sources of information are not freely available, such as in languages other than English [7,9,19,20,22,38,43,47]; • when researchers investigate questions related to patients and population, while these questions are not discussed with medical doctors or require large population samples. We can mention for instance sentiment analysis on medication and vaccines [7,20,33,38,43,47], and adverse drug effects [76,77]; • when mental health of patients is concerned in cases like depression [9, 4, 78], eating disorders [79], suicide detection and prevention [19,80], quality of life of patients [31,81], and drug misuse [82][83][84].…”