Localizing and Quantifying Damage in Social Media Images

Abstract

Traditional post-disaster assessment of damage heavily relies on expensive GIS data, especially remote sensing image data. In recent years, social media has become a rich source of disaster information that may be useful in assessing damage at a lower cost. Such information includes text (e.g., tweets) or images posted by eyewitnesses of a disaster. Most of the existing research explores the use of text in identifying situational awareness information useful for disaster response teams. The use of social media images to assess disaster damage is limited. In this paper, we propose a novel approach, based on convolutional neural networks and class activation maps, to locate damage in a disaster image and to quantify the degree of the damage. Our proposed approach enables the use of social network images for post-disaster damage assessment, and provides an inexpensive and feasible alternative to the more expensive GIS approach.

Critical Image Identification via Incident-Type Definition Using Smartphone Data during an Emergency: A Case Study of the 2020 Heavy Rainfall Event in Korea

“…Alam et al [35] proposed an image-processing system called Image4Act, which was based on the visual geometry group-16 (VGG-16) model, and sought to support relief efforts by collecting, removing noise from, and classifying images posted on social media. Li et al [36] proposed a method to create a damage detection map based on the VGG-19 model and to measure its severity for images collected from social media. Weber et al [16] trained the Resnet-18 model to classify images collected from social media into 43 types of incidents using a significant number of class-positive and class-negative datasets, and it could automatically detect incidents from images collected from social media.…”

Detection of Disaster-Affected Cultural Heritage Sites from Social Media Images Using Deep Learning Techniques

“…Later, Nguyen et al investigated a more generic solution to classify disaster images according to damage severity using convolutional neural networks [69]. Similarly, Li et al proposed a method based on class activation mapping to localize and quantify damage in social media images posted during disasters [57]. Taking a step further, Li et al explored domain adaptation approach to identify disaster damage images during an emergent event when there is scarcity of labeled data [56].…”

HeteroEdge

“…A large set of social sensing applications are sensitive to delay, i.e., have real-time requirements. Examples of such applications include intelligent transportation systems [48], environmental sensing [46], and disaster and emergency response [16]. Traditional social sensing applications push all the computation tasks to the remote servers/cloud, which can be quite ineffective, particularly for delay-sensitive applications, when the network bandwidth is limited and the communication latency is high [47,49].…”

“…Tasks for DDA: i) edge devices equipped with cameras (e.g., dash cameras, UAVs) are tasked to capture live images of locations of interest; ii) extracting Damage Detection Map (DDM) features using Convolutional Neural Network (CNN) model from raw images; iii) assess damage severity from DDM using the algorithm in [16].…”