Predicting Flu Trends using Twitter data

Abstract

Abstract-Reducing the impact of seasonal influenza epidemics and other pandemics such as the H1N1 is of paramount importance for public health authorities. Studies have shown that effective interventions can be taken to contain the epidemics if early detection can be made. Traditional approach employed by the Centers for Disease Control and Prevention (CDC) includes collecting influenza-like illness (ILI) activity data from "sentinel" medical practices. Typically there is a 1-2 week delay between the time a patient is diagnosed and the moment that data point becomes available in aggregate ILI reports. In this paper we present the Social Network Enabled Flu Trends (SNEFT) framework, which monitors messages posted on Twitter with a mention of flu indicators to track and predict the emergence and spread of an influenza epidemic in a population. Based on the data collected during 2009 and 2010, we find that the volume of flu related tweets is highly correlated with the number of ILI cases reported by CDC. We further devise auto-regression models to predict the ILI activity level in a population. The models predict data collected and published by CDC, as the percentage of visits to "sentinel" physicians attributable to ILI in successively weeks. We test models with previous CDC data, with and without measures of Twitter data, showing that Twitter data can substantially improve the models prediction accuracy. Therefore, Twitter data provides real-time assessment of ILI activity.

Online Social Networks Flu Trend Tracker: A Novel Sensory Approach to Predict Flu Trends

“…In this paper, we extend our preliminary analysis [1,2], and provide a continuing study of using OSN's to track the emergence and spread of seasonal flu in the year 2010-2011. OSN data which demonstrated high correlation with CDC ILI rate for the year 2009-2010, was affected by spurious messages and so text mining techniques were applied.…”

“…Initial analysis for the period 2009-2010 indicated a strong correlation between CDC and Twitter data on the flu incidences [1]. However results for the year 2010-2011 showed a significant drop in the correlation coefficient from 0.98 to 0.47.…”

“…Tweetminster, a media utility tool design to make UK politics open and social, analyses political tweets, to establish the correlations between buzz on Twitter and election results. In June 2010, we introduced the SNEFT architecture as a continuous data collection engine which combines the detection and prediction capability on social networks in discovering real world flu trends [1,2,4].…”

“…[1]. For 2010-2011, we have 4.5 million tweets from 1.9 million unique users and 2.0 million facebook posts from 1.5 million unique facebook users.…”

A Spatio-Temporal Approach to the Discovery of Online Social Trends

“…Technology companies may use OSNs to obtain user feedbacks when rolling out new products. Moreover, OSNs may also be used to monitor public health trends, including the outbreak of seasonal influenza or the H1N1 Swine flu [1].…”