Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic

Abstract

BackgroundGoogle Flu Trends (GFT) uses anonymized, aggregated internet search activity to provide near-real time estimates of influenza activity. GFT estimates have shown a strong correlation with official influenza surveillance data. The 2009 influenza virus A (H1N1) pandemic [pH1N1] provided the first opportunity to evaluate GFT during a non-seasonal influenza outbreak. In September 2009, an updated United States GFT model was developed using data from the beginning of pH1N1.Methodology/Principal FindingsWe evaluated the accuracy of each U.S. GFT model by comparing weekly estimates of ILI (influenza-like illness) activity with the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet). For each GFT model we calculated the correlation and RMSE (root mean square error) between model estimates and ILINet for four time periods: pre-H1N1, Summer H1N1, Winter H1N1, and H1N1 overall (Mar 2009–Dec 2009). We also compared the number of queries, query volume, and types of queries (e.g., influenza symptoms, influenza complications) in each model. Both models' estimates were highly correlated with ILINet pre-H1N1 and over the entire surveillance period, although the original model underestimated the magnitude of ILI activity during pH1N1. The updated model was more correlated with ILINet than the original model during Summer H1N1 (r = 0.95 and 0.29, respectively). The updated model included more search query terms than the original model, with more queries directly related to influenza infection, whereas the original model contained more queries related to influenza complications.ConclusionsInternet search behavior changed during pH1N1, particularly in the categories “influenza complications” and “term for influenza.” The complications associated with pH1N1, the fact that pH1N1 began in the summer rather than winter, and changes in health-seeking behavior each may have played a part. Both GFT models performed well prior to and during pH1N1, although the updated model performed better during pH1N1, especially during the summer months.

Accurate estimation of influenza epidemics using Google search data via ARGO

“…However, significant discrepancies between GFT's flu estimates and those measured by the Centers for Disease Control (CDC) in subsequent years led to considerable doubt about the value of digital disease detection systems (13). Although multiple articles have identified methodological flaws in GFT's original algorithm (14)(15)(16) and have led to incremental improvements (14,16) (see also googleresearch. blogspot.com/2014/10/google-flu-trends-gets-brand-new-engine.html), a statistical framework that is theoretically sound and capable of accurate estimation is still lacking.…”

Big Data in Organizations and the Role of Human Resource Management

“…They identified 1,152 data points that related to the flu (Ginsberg et al 2009), however, they initially did not seek new or abnormal search patterns like the A-H1N1 influenza (Cook et al 2011, Olson et al 2013. Those inconsistencies within the risk network caused Google Flu to overestimate flu prevalence, making the results no longer precise, and rendering them even less accurate than those of the CDC (Lazer et al 2014, Kugler 2016.…”

Google Flu Trends Still Appears Sick: An Evaluation of the 2013-2014 Flu Season

“…Science is a cumulative enterprise, and progress requires the ability for the community to continually assess the work on which they are building (6,7). GFT has not been very forthcoming with this information in the past, going so far as to release misleading example search terms in previous publications (2,3,8).…”

“…Science is a cumulative enterprise, and progress requires the ability for the community to continually assess the work on which they are building (6, 7). GFT has not been very forthcoming with this information in the past, going so far as to release misleading example search terms in previous publications (2,3,8).These transparency problems have, if anything, become worse. While the data on the intensity of media coverage of flu outbreaks does not involve privacy concerns, GFT has not released this data nor have they provided an explanation of how the information was collected and utilized.…”