Twitter-Based Influenza Detection After Flu Peak via Tweets With Indirect Information: Text Mining Study

Abstract

BackgroundThe recent rise in popularity and scale of social networking services (SNSs) has resulted in an increasing need for SNS-based information extraction systems. A popular application of SNS data is health surveillance for predicting an outbreak of epidemics by detecting diseases from text messages posted on SNS platforms. Such applications share the following logic: they incorporate SNS users as social sensors. These social sensor–based approaches also share a common problem: SNS-based surveillance are much more reliable if sufficient numbers of users are active, and small or inactive populations produce inconsistent results.ObjectiveThis study proposes a novel approach to estimate the trend of patient numbers using indirect information covering both urban areas and rural areas within the posts.MethodsWe presented a TRAP model by embedding both direct information and indirect information. A collection of tweets spanning 3 years (7 million influenza-related tweets in Japanese) was used to evaluate the model. Both direct information and indirect information that mention other places were used. As indirect information is less reliable (too noisy or too old) than direct information, the indirect information data were not used directly and were considered as inhibiting direct information. For example, when indirect information appeared often, it was considered as signifying that everyone already had a known disease, leading to a small amount of direct information.ResultsThe estimation performance of our approach was evaluated using the correlation coefficient between the number of influenza cases as the gold standard values and the estimated values by the proposed models. The results revealed that the baseline model (BASELINE+NLP) shows .36 and that the proposed model (TRAP+NLP) improved the accuracy (.70, +.34 points).ConclusionsThe proposed approach by which the indirect information inhibits direct information exhibited improved estimation performance not only in rural cities but also in urban cities, which demonstrated the effectiveness of the proposed method consisting of a TRAP model and natural language processing (NLP) classification.

Recent Advances in Using Natural Language Processing to Address Public Health Research Questions Using Social Media and ConsumerGenerated Data

“…The vast majority of the papers reviewed focussed on analysing English language text (68 papers), with two papers focussing on Chinese text [76,77] and one paper focussing on Japanese text [31]. With respect to the geographical location of first authors, most of the articles emerged from North America (55), with Europe 7, and Asia (including Australasia and Turkey) (6) all represented.…”

“…Of the six papers reviewed (see Table 4), four used Twitter data [31][32][33]57], and two used Reddit data [10,14], while Al-Garadi et al, provided a review that concentrated on Twitter and Weibo, the Chinese language microblog service [32]. Two of the papers reviewed described the use of supervised machine learning methods [31,32], three papers used unsupervised machine learning methods [10,14,32], and one used a lexicon-based approach [57]. Machine learning methods were used to perform a variety of tasks, including surveillance [10,14,[31][32][33]57], health communication [32], and sentiment analysis [32].…”

“…Two of the papers reviewed described the use of supervised machine learning methods [31,32], three papers used unsupervised machine learning methods [10,14,32], and one used a lexicon-based approach [57]. Machine learning methods were used to perform a variety of tasks, including surveillance [10,14,[31][32][33]57], health communication [32], and sentiment analysis [32]. Several studies concentrated on influenza surveillance using English [10,33] and Japanese [31] Twitter data.…”

“…Machine learning methods were used to perform a variety of tasks, including surveillance [10,14,[31][32][33]57], health communication [32], and sentiment analysis [32]. Several studies concentrated on influenza surveillance using English [10,33] and Japanese [31] Twitter data.…”

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

“…The experiments' results demonstrated the practicability of the proposed approach, which showed acceptable correlation comparing with medical reports statistics, especially at the outbreak and early spread (early epidemic) stage. The authors extended their work in [67] and implemented a robust influenza prediction model that enabled the use of direct and indirect information using tweets from urban and rural areas in Japan. This work was further extended in [68].…”