Prototyping a Social Media Flooding Photo Screening System Based on Deep Learning

Ning, Huan; Li, Zhenlong; Hodgson, Michael E.; Wang, Cuizhen

doi:10.3390/ijgi9020104

Cited by 44 publications

(16 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ontologies applied within flooding to incorporate sensor data, although available, are limited [105][106][107][108] and applied in specific scenarios, as noted in a recent systematic review of flooding ontologies [109]. However, the application of deep learning and semantic web in disaster response has been limited, primarily aimed at classification and identification of disaster-related information in social media [110][111][112] or analysing remote sensing [113] and aerial imagery [114]. The use of semantic technologies in smart cities has led to discovering new opportunities such as information discovery, categorisation of events, complex event processing and reasoning for decision making, as the semantic networks provide a powerful way of transforming knowledge into machine-readable content [115].…”

Section: Literature Reviewmentioning

confidence: 99%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

Traditional Artificial Intelligence (AI) technologies used in developing smart cities solutions, Machine Learning (ML) and recently Deep Learning (DL), rely more on utilising best representative training datasets and features engineering and less on the available domain expertise. We argue that such an approach to solution development makes the outcome of solutions less explainable, i.e., it is often not possible to explain the results of the model. There is a growing concern among policymakers in cities with this lack of explainability of AI solutions, and this is considered a major hindrance in the wider acceptability and trust in such AI-based solutions. In this work, we survey the concept of ‘explainable deep learning’ as a subset of the ‘explainable AI’ problem and propose a new solution using Semantic Web technologies, demonstrated with a smart cities flood monitoring application in the context of a European Commission-funded project. Monitoring of gullies and drainage in crucial geographical areas susceptible to flooding issues is an important aspect of any flood monitoring solution. Typical solutions for this problem involve the use of cameras to capture images showing the affected areas in real-time with different objects such as leaves, plastic bottles etc., and building a DL-based classifier to detect such objects and classify blockages based on the presence and coverage of these objects in the images. In this work, we uniquely propose an Explainable AI solution using DL and Semantic Web technologies to build a hybrid classifier. In this hybrid classifier, the DL component detects object presence and coverage level and semantic rules designed with close consultation with experts carry out the classification. By using the expert knowledge in the flooding context, our hybrid classifier provides the flexibility on categorising the image using objects and their coverage relationships. The experimental results demonstrated with a real-world use case showed that this hybrid approach of image classification has on average 11% improvement (F-Measure) in image classification performance compared to DL-only classifier. It also has the distinct advantage of integrating experts’ knowledge on defining the decision-making rules to represent the complex circumstances and using such knowledge to explain the results.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The vast human-sensed information captured by social media has not yet been fully utilized for smart city planning, management, and engagement [27]. Several frameworks exist to leverage useful event monitoring information from social media data, such as text classification methods to generate incident intelligence reports from tweets [28] and rapid flood mapping based on social media photos [29]. Social media platforms can provide real-time feedback for communities affected by hazards through crowdsourced damage reports, wellness checks, and capabilities to seek help and respond to the needs of victims [30].…”

Section: Social Media Use In Resilient Smart Citiesmentioning

confidence: 99%

Use of Social Media to Seek and Provide Help in Hurricanes Florence and Michael

DiCarlo

Berglund

2020

Smart Cities

View full text Add to dashboard Cite

During hazardous events, communities can use existing social media networks to share information in real time and initiate a local disaster response. This research conducted a web-based survey to explore two behaviors around the use of social media during hurricanes: seeking help and responding to help requests. Through the survey, we sampled 434 individuals across several counties affected by 2018 hurricanes Florence and Michael, which were both designated by the National Oceanic and Atmospheric Administration as billion-dollar weather disasters. The survey questions collected data about demographics, social media use habits, perceptions towards social media, hurricane damages, and actions taken during a hurricane to seek and provide help. The Theory of Planned Behavior (TPB) was used to conceptualize and frame parameters that affect intentions and behaviors regarding the use of social media during hurricanes to seek and provide help. Survey responses are analyzed using statistical regression to evaluate hypotheses about the influence of factors on seeking help and responding to help requests. Regression analyses indicate that attitude and perceived behavioral control predict intention to access social media during a hurricane, partially supporting the TPB. Intention and experiencing urgent damages predict help-seeking behaviors using social media. Posting frequency to social media under normal conditions and the number of help requests seen during the event predict help-responding behaviors. Linear regression equations governing intention and behavior were parameterized using survey results. The factors underlying social media behavior during hurricanes as identified in this research provide insight for understanding how smart information technologies, such as personal devices and social media networks, support community self-sufficiency and hazard resilience.

show abstract

“…In such cases, assuming that systems may be fully automated or semi-automated, some measures can be adopted to reduce the probability of false alarms, such as the use of multiple cameras or data mining to support decisions. In this sense, since the acquisition of images and videos can also be performed from different sources of data, social media could be employed as a complementary source of data when detecting events, exploiting the idea that people may act as independent sensors in smart cities [51,52]. Moreover, for semi-automated systems, any detected event could require validation by a human, increasing accuracy at the cost of higher decision times for the systems.…”

Section: Practical Issues When Employing Cameras For Emergency Alertingmentioning

confidence: 99%

On the Use of Cameras for the Detection of Critical Events in Sensors-Based Emergency Alerting Systems

Costa

Vasques

Portugal

et al. 2020

JSAN

View full text Add to dashboard Cite

The adoption of emergency alerting systems can bring countless benefits when managing urban areas, industrial plants, farms, roads and virtually any area that is subject to the occurrence of critical events, supporting in rescue operations and reducing their negative impacts. For such systems, a promising approach is to exploit scalar sensors to detect events of interest, allowing for the distributed monitoring of different variables. However, the use of cameras as visual sensors can enhance the detection of critical events, which can be employed along with scalar sensors for a more comprehensive perception of the environment. Although the particularities of visual sensing may be challenging in some scenarios, the combination of scalar and visual sensors for the early detection of emergency situations can be valuable for many scenarios, such as smart cities and industry 4.0, bringing promising results. Therefore, in this article, we extend a sensors-based emergency detection and alerting system to also exploit visual monitoring when identifying critical events. Implementation and experimental details are provided to reinforce the use of cameras as a relevant sensor unit, bringing promising results for emergencies management.

show abstract

Prototyping a Social Media Flooding Photo Screening System Based on Deep Learning

Cited by 44 publications

References 46 publications

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Use of Social Media to Seek and Provide Help in Hurricanes Florence and Michael

On the Use of Cameras for the Detection of Critical Events in Sensors-Based Emergency Alerting Systems

Contact Info

Product

Resources

About