Toward Using Twitter for Tracking COVID-19: A Natural Language Processing Pipeline and Exploratory Data Set

Klein, Ari Z.; Magge, Arjun; O’Connor, Karen; Amaro, Jesus Ivan Flores; Weissenbacher, Davy; Gonzalez-Hernandez, Graciela

doi:10.2196/25314

Cited by 53 publications

(32 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shimkhada et al [ 27 ] used Twitter Chat to identify barriers and responsive policy of patients with metastatic breast cancer care. Klein et al [ 28 ] conducted Twitter data to track the spread of COVID-19. Johnson et al [ 29 ] performed Google Trends to monitor sexually transmitted infections Chicago.…”

Section: Discussionmentioning

confidence: 99%

Predicting Norovirus in the United States Using Google Trends: Infodemiology Study

et al. 2021

View full text Add to dashboard Cite

Background Norovirus is a contagious disease. The transmission of norovirus spreads quickly and easily in various ways. Because effective methods to prevent or treat norovirus have not been discovered, it is important to rapidly recognize and report norovirus outbreaks in the early phase. Internet search has been a useful method for people to access information immediately. With the precise record of internet search trends, internet search has been a useful tool to manifest infectious disease outbreaks. Objective In this study, we tried to discover the correlation between internet search terms and norovirus infection. Methods The internet search trend data of norovirus were obtained from Google Trends. We used cross-correlation analysis to discover the temporal correlation between norovirus and other terms. We also used multiple linear regression with the stepwise method to recognize the most important predictors of internet search trends and norovirus. In addition, we evaluated the temporal correlation between actual norovirus cases and internet search terms in New York, California, and the United States as a whole. Results Some Google search terms such as gastroenteritis, watery diarrhea, and stomach bug coincided with norovirus Google Trends. Some Google search terms such as contagious, travel, and party presented earlier than norovirus Google Trends. Some Google search terms such as dehydration, bar, and coronavirus presented several months later than norovirus Google Trends. We found that fever, gastroenteritis, poison, cruise, wedding, and watery diarrhea were important factors correlated with norovirus Google Trends. In actual norovirus cases from New York, California, and the United States as a whole, some Google search terms presented with, earlier, or later than actual norovirus cases. Conclusions Our study provides novel strategy-based internet search evidence regarding the epidemiology of norovirus.

show abstract

Section: Discussionmentioning

confidence: 99%

Predicting Norovirus in the United States Using Google Trends: Infodemiology Study

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Teams placing second and third achieved F 1 -scores of 0.77 and 0.76, respectively, using COVID-Twitter-BERT, while the teams (that submitted system descriptions) that achieved F 1 -scores of less than 0.76 did not use models pre-trained on tweets related to COVID-19. The leading team outperformed a benchmark classifier presented in recent work (Klein et al, 2021), which was based on COVID-Twitter-BERT and achieved an F 1 -score (0.76) similar to that of the teams placing second and third.…”

Section: Task 5: Classification Of Tweets Self-reporting Potential Cases Of Covid-19mentioning

confidence: 60%

“…Task 5 is a binary classification task that involves automatically distinguishing tweets that self-report potential cases of COVID-19 ("potential case" tweets) from those that do not ("other" tweets), where "potential case" tweets broadly include those indicating that the user or a member of the user's household was denied testing for COVID-19, showing symptoms of COVID-19, potentially exposed to cases of COVID-19, or had had experiences that pose a higher risk of exposure to COVID-19. The training set (Klein et al, 2021) contains 7181 tweets: 1148 (16%) "potential case" tweets (annotated as "1") and 6033 (84%) "other" tweets (annotated as "0"). The test set contains 1795 annotated tweets: 308 (17%) "potential case" tweets and 1487 (83%) "other" tweets.…”

Section: Task 5: Classification Of Tweets Self-reporting Potential Cases Of Covid-19mentioning

confidence: 99%

Overview of the Sixth Social Media Mining for Health Applications (#SMM4H) Shared Tasks at NAACL 2021

Magge¹,

Klein²,

Miranda-Escalada³

et al. 2021

Proceedings of the Sixth Social Media Mining for Health (#SMM4H) Workshop and Shared Task

Self Cite

View full text Add to dashboard Cite

The global growth of social media usage over the past decade has opened research avenues for mining health related information that can ultimately be used to improve public health. The Social Media Mining for Health Research and Applications (#SMM4H) shared tasks in its sixth iteration sought to advance the use of social media texts such as Twitter for pharmacovigilance, disease tracking and patient centered outcomes. #SMM4H 2021 hosted a total of eight tasks that included reruns of adverse drug effect extraction in English and Russian and newer tasks such as detecting medication non-adherence from Twitter and WebMD forum, detecting self-reported adverse pregnancy outcomes, detecting cases and symptoms of COVID-19, identifying occupations mentioned in Spanish by Twitter users, and detecting self-reported breast cancer diagnosis. The eight tasks included a total of 12 individual subtasks spanning three languages requiring methods for binary classification, multiclass classification, named entity recognition (NER) and entity normalization. With a total of 97 registering teams and 40 teams submitting predictions, the interest in the shared tasks grew by 70% and participation grew by 38% compared to the previous iteration.

show abstract

“…However, this approach can be applied to other search engines (eg, Baidu, Yahoo, Naver), as was done in previous studies on different diseases and on COVID-19 in Hubei province, China [24,25]. Previous studies have shown that self-reported symptoms on social media networks, such as Twitter, can provide useful information to track the COVID-19 pandemic and can be used for infoveillance along with search engine data [26][27][28]. Our approach could, in principle, be applied to social media as well.…”

Section: Discussionmentioning

confidence: 99%

Forecasting the COVID-19 Epidemic By Integrating Symptom Search Behavior Into Predictive Models: Infoveillance Study

Rabiolo¹,

Alladio²,

Morales³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Previous studies have suggested associations between trends of web searches and COVID-19 traditional metrics. It remains unclear whether models incorporating trends of digital searches lead to better predictions. Objective The aim of this study is to investigate the relationship between Google Trends searches of symptoms associated with COVID-19 and confirmed COVID-19 cases and deaths. We aim to develop predictive models to forecast the COVID-19 epidemic based on a combination of Google Trends searches of symptoms and conventional COVID-19 metrics. Methods An open-access web application was developed to evaluate Google Trends and traditional COVID-19 metrics via an interactive framework based on principal component analysis (PCA) and time series modeling. The application facilitates the analysis of symptom search behavior associated with COVID-19 disease in 188 countries. In this study, we selected the data of nine countries as case studies to represent all continents. PCA was used to perform data dimensionality reduction, and three different time series models (error, trend, seasonality; autoregressive integrated moving average; and feed-forward neural network autoregression) were used to predict COVID-19 metrics in the upcoming 14 days. The models were compared in terms of prediction ability using the root mean square error (RMSE) of the first principal component (PC1). The predictive abilities of models generated with both Google Trends data and conventional COVID-19 metrics were compared with those fitted with conventional COVID-19 metrics only. Results The degree of correlation and the best time lag varied as a function of the selected country and topic searched; in general, the optimal time lag was within 15 days. Overall, predictions of PC1 based on both search terms and COVID-19 traditional metrics performed better than those not including Google searches (median 1.56, IQR 0.90-2.49 versus median 1.87, IQR 1.09-2.95, respectively), but the improvement in prediction varied as a function of the selected country and time frame. The best model varied as a function of country, time range, and period of time selected. Models based on a 7-day moving average led to considerably smaller RMSE values as opposed to those calculated with raw data (median 0.90, IQR 0.50-1.53 versus median 2.27, IQR 1.62-3.74, respectively). Conclusions The inclusion of digital online searches in statistical models may improve the nowcasting and forecasting of the COVID-19 epidemic and could be used as one of the surveillance systems of COVID-19 disease. We provide a free web application operating with nearly real-time data that anyone can use to make predictions of outbreaks, improve estimates of the dynamics of ongoing epidemics, and predict future or rebound waves.

show abstract

Toward Using Twitter for Tracking COVID-19: A Natural Language Processing Pipeline and Exploratory Data Set

Cited by 53 publications

References 12 publications

Predicting Norovirus in the United States Using Google Trends: Infodemiology Study

Predicting Norovirus in the United States Using Google Trends: Infodemiology Study

Overview of the Sixth Social Media Mining for Health Applications (#SMM4H) Shared Tasks at NAACL 2021

Forecasting the COVID-19 Epidemic By Integrating Symptom Search Behavior Into Predictive Models: Infoveillance Study

Contact Info

Product

Resources

About