Data mining Twitter during the UK floods: Investigating the potential use of social media in emergency management

Spielhofer, Thomas; Greenlaw, Reynold; Markham, Deborah; Hahne, Anna Sophie

doi:10.1109/ict-dm.2016.7857213

“…They limited their research on New Zealand and Australia and on earthquakes and used a keyword-based approach which has clear limits in expanding to other events or languages or collecting all available information. Spielhofer, Greenlaw, Markham, and Hahne (2016) present another method for analyzing social media streams. While their noise reduction algorithm is highly valuable, it is only the basis for further (geospatial) analysis, which we aim to cover in our research.…”

Section: Related Workmentioning

confidence: 99%

Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment

Resch

¹

,

Usländer

²

,

Havas

³

2017

Cartography and Geographic Information Science

View full text Add to dashboard Cite

Current disaster management procedures to cope with human and economic losses and to manage a disaster's aftermath suffer from a number of shortcomings like high temporal lags or limited temporal and spatial resolution. This paper presents an approach to analyze social media posts to assess the footprint of and the damage caused by natural disasters through combining machinelearning techniques (Latent Dirichlet Allocation) for semantic information extraction with spatial and temporal analysis (local spatial autocorrelation) for hot spot detection. Our results demonstrate that earthquake footprints can be reliably and accurately identified in our use case. More, a number of relevant semantic topics can be automatically identified without a priori knowledge, revealing clearly differing temporal and spatial signatures. Furthermore, we are able to generate a damage map that indicates where significant losses have occurred. The validation of our results using statistical measures, complemented by the official earthquake footprint by US Geological Survey and the results of the HAZUS loss model, shows that our approach produces valid and reliable outputs. Thus, our approach may improve current disaster management procedures through generating a new and unseen information layer in near real time.

show abstract

“…In a study classifying flood-relevant UK tweets (Spielhofer et al, 2016), Naive Bayes performed worse than Logistic Regression with a class-imbalanced dataset (as our dataset exhibited). Next, we trained the classifier model, and used the unseen test set of tweets to evaluate its performance.…”

Section: 23mentioning

confidence: 77%

“…One exception was the integration of environmental sensor data with catchment polygons to prioritise tweets in Sao Paulo (de Assis et al, 2016). Existing web applications-that harness social media during emergency events-using machine learning techniques have generally been developed for specific incidents to be searched for -typically, one-off catchment to regional scale situations (Spielhofer et al, 2016) -rather than supporting national level surveillance monitoring of natural hazard events and their social impacts, e.g. only at a city-scale (Eilander et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

Development of a national-scale real-time Twitter data mining pipeline for social geodata on the potential impacts of flooding on communities

Barker

¹

,

Macleod

²

2019

Environmental Modelling & Software

View full text Add to dashboard Cite

Social media, particularly Twitter, is increasingly used to improve resilience during extreme weather events/emergency management situations, including floods: by communicating potential risks and their impacts, and informing agencies and responders. In this paper, we developed a prototype national-scale Twitter data mining pipeline for improved stakeholder situational awareness during flooding events across Great Britain, by retrieving relevant social geodata, grounded in environmental data sources (flood warnings and river levels). With potential users we identified and addressed three research questions to develop this application, whose components constitute a modular architecture for real-time dashboards. First, polling national flood warning and river level Web data sources to obtain at-risk locations. Secondly, real-time retrieval of geotagged tweets, proximate to at-risk areas. Thirdly, filtering flood-relevant tweets with natural language processing and machine learning libraries, using word embeddings of tweets. We demonstrated the national-scale social geodata pipeline using over 420,000 georeferenced tweets obtained between 20-29th June 2016. Highlights • Prototype real-time social geodata pipeline for flood events and demonstration dataset • National-scale flood warnings/river levels set 'at-risk areas' in Twitter API queries • Monitoring multiple locations (without keywords) retrieved current, geotagged tweets • Novel application of word embeddings in flooding context identified relevant tweets • Pipeline extracts tweets to visualise using open-source libraries (SciKit Learn/Gensim)

show abstract

“…for specific incidents to be searched for -typically, one-off catchment to regional scale situations (Spielhofer et al, 2016) -rather than supporting national level surveillance monitoring of natural hazard events and their social impacts, e.g. only at a city-scale (Eilander et al, 2016).…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Development of a national-scale real-time Twitter data mining pipeline for social geodata on the potential impacts of flooding on communities

Barker¹,

MacLeod²

2019

Preprint

1

0

View full text Add to dashboard Cite

Social media, particularly Twitter, is increasingly used to improve resilience during extreme weather events/emergency management situations, including floods: by communicating potential risks and their impacts, and informing agencies and responders. In this paper, we developed a prototype national-scale Twitter data mining pipeline for improved stakeholder situational awareness during flooding events across Great Britain, by retrieving relevant social geodata, grounded in environmental data sources (flood warnings and river levels). With potential users we identified and addressed three research questions to develop this application, whose components constitute a modular architecture for real-time dashboards. First, polling national flood warning and river level Web data sources to obtain at-risk locations. Secondly, real-time retrieval of geotagged tweets, proximate to at-risk areas. Thirdly, filtering flood-relevant tweets with natural language processing and machine learning libraries, using word embeddings of tweets. We demonstrated the national-scale social geodata pipeline using over 420,000 georeferenced tweets obtained between 20 and 29th June 2016.

show abstract

Data mining Twitter during the UK floods: Investigating the potential use of social media in emergency management

Cited by 28 publications

References 6 publications

Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment

Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment

Development of a national-scale real-time Twitter data mining pipeline for social geodata on the potential impacts of flooding on communities

Development of a national-scale real-time Twitter data mining pipeline for social geodata on the potential impacts of flooding on communities

Contact Info

Product

Resources

About