Damage Assessment from Social Media Imagery Data During Disasters

Nguyen, Dat T.; Ofli, Ferda; Imran, Muhammad; Mitra, Prasenjit

doi:10.1145/3110025.3110109

Cited by 134 publications

(113 citation statements)

References 23 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…Despite the extensive use of machine learning tools for analyzing social media text data posted during disaster events, there is not much work on analyzing social media images posted by eyewitnesses of a disaster. One pioneering work in this area [12], used trained CNN models, specifically, VGG16 networks fine-tuned on the disaster image datasets that are also used in our work (see Table I), and showed that the CNN models perform better than standard techniques based on bags-of-visual-words. Our CNN results, using VGG19 finetuned on the same disaster image datasets, are similar to those reported in [12], except for Matthew Hurricane, for which the dataset is relatively small and the model can't be trained well.…”

Section: Related Workmentioning

confidence: 97%

“…Thus, social media images can serve as an ancillary yet rich source of visual information in disaster damage assessment. Pioneering works with focus on the utility of social media images in disaster response include [11], [12], where the goal is to use convolutional neural networks (CNN) to assess the severity of the damage (specifically, to classify social media images based on the degree of the damage as: severe, mild, and none).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Localizing and Quantifying Damage in Social Media Images

Caragea

Zhang

et al. 2018

2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)

Self Cite

View full text Add to dashboard Cite

Traditional post-disaster assessment of damage heavily relies on expensive GIS data, especially remote sensing image data. In recent years, social media has become a rich source of disaster information that may be useful in assessing damage at a lower cost. Such information includes text (e.g., tweets) or images posted by eyewitnesses of a disaster. Most of the existing research explores the use of text in identifying situational awareness information useful for disaster response teams. The use of social media images to assess disaster damage is limited. In this paper, we propose a novel approach, based on convolutional neural networks and class activation maps, to locate damage in a disaster image and to quantify the degree of the damage. Our proposed approach enables the use of social network images for post-disaster damage assessment, and provides an inexpensive and feasible alternative to the more expensive GIS approach.

show abstract

Section: Related Workmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Localizing and Quantifying Damage in Social Media Images

Caragea

Zhang

et al. 2018

2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, understanding the extent of the infrastructure and utility damage caused by a disaster is one of the core situational awareness tasks listed earlier. Several studies in the literature have already shown that social media images can be analysed for automatic damage assessment in addition to the textual content analysis (Liang, Caverlee, and Mander 2013a;Daly and Thom 2016;Lagerstrom et al 2016;Nguyen et al 2017c). Inspired by these studies, we perform an infrastructural damage assessment task on cleaned social media imagery content.…”

Section: Extracting Useful Informationmentioning

confidence: 99%

Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of Hurricanes Harvey, Irma, and Maria

Alam

Ofli

Imran

2019

Behaviour & Information Technology

Self Cite

View full text Add to dashboard Cite

People increasingly use microblogging platforms such as Twitter during natural disasters and emergencies. Research studies have revealed the usefulness of the data available on Twitter for several disaster response tasks. However, making sense of social media data is a challenging task due to several reasons such as limitations of available tools to analyse high-volume and highvelocity data streams, dealing with information overload, among others. To eliminate such limitations, in this work, we first show that textual and imagery content on social media provide complementary information useful to improve situational awareness. We then explore ways in which various Artificial Intelligence techniques from Natural Language Processing and Computer Vision fields can exploit such complementary information generated during disaster events. Finally, we propose a methodological approach that combines several computational techniques effectively in a unified framework to help humanitarian organisations in their relief efforts. We conduct extensive experiments using textual and imagery content from millions of tweets posted during the three major disaster events in the 2017 Atlantic Hurricane season. Our study reveals that the distributions of various types of useful information can inform crisis managers and responders and facilitate the development of future automated systems for disaster management.

show abstract

“…For example, [42], [43] tackle with fire detection whereas [44] addresses flood detection from social media images. Furthermore, [45] proposes an automatic image processing pipeline for social media imagery data, and [46], [47] further explore infrastructural damage assessment problem, mainly at the image classification level.…”

Section: Related Workmentioning

confidence: 99%

Nazr-CNN: Fine-Grained Classification of UAV Imagery for Damage Assessment

Attari

Ofli

Awad

et al. 2017

2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA)

Self Cite

View full text Add to dashboard Cite

We propose Nazr-CNN 1 , a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage.To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pretrained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines. 1 Nazr means "sight" in Arabic. 2

show abstract

Damage Assessment from Social Media Imagery Data During Disasters

Cited by 134 publications

References 23 publications

Localizing and Quantifying Damage in Social Media Images

Localizing and Quantifying Damage in Social Media Images

Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of Hurricanes Harvey, Irma, and Maria

Nazr-CNN: Fine-Grained Classification of UAV Imagery for Damage Assessment

Contact Info

Product

Resources

About