Deep learning-based aerial image segmentation with open data for disaster impact assessment

Gupta, Ananya; Watson, Simon; Yin, Hujun

doi:10.1016/j.neucom.2020.02.139

Cited by 38 publications

(23 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some news values require face recognition (Zhu et al, 2021 ) and facial analysis (Li and Deng, 2022 ), which can be efficiently done with the use of CNN-based face detection and facial expression prediction. For identifying the impact of events in images, deep networks can be applied for a disaster impact assessment on aerial imagery (Gupta et al, 2021 ) and image-text pairs of tweets (Rizk et al, 2019 ). Lastly, there are a few works where a BERT-based model has been used for detecting racist (Fokkens et al, 2018 ), gender (Chiril et al, 2021 ) and immigrant (Sánchez-Junquera et al, 2021 ) stereotypes in news, tweets and political debates respectively.…”

Section: Discussionmentioning

confidence: 99%

Understanding image-text relations and news values for multimodal news analysis

Cheema

Hakimov

Müller-Budack

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

The analysis of news dissemination is of utmost importance since the credibility of information and the identification of disinformation and misinformation affect society as a whole. Given the large amounts of news data published daily on the Web, the empirical analysis of news with regard to research questions and the detection of problematic news content on the Web require computational methods that work at scale. Today's online news are typically disseminated in a multimodal form, including various presentation modalities such as text, image, audio, and video. Recent developments in multimodal machine learning now make it possible to capture basic “descriptive” relations between modalities–such as correspondences between words and phrases, on the one hand, and corresponding visual depictions of the verbally expressed information on the other. Although such advances have enabled tremendous progress in tasks like image captioning, text-to-image generation and visual question answering, in domains such as news dissemination, there is a need to go further. In this paper, we introduce a novel framework for the computational analysis of multimodal news. We motivate a set of more complex image-text relations as well as multimodal news values based on real examples of news reports and consider their realization by computational approaches. To this end, we provide (a) an overview of existing literature from semiotics where detailed proposals have been made for taxonomies covering diverse image-text relations generalisable to any domain; (b) an overview of computational work that derives models of image-text relations from data; and (c) an overview of a particular class of news-centric attributes developed in journalism studies called news values. The result is a novel framework for multimodal news analysis that closes existing gaps in previous work while maintaining and combining the strengths of those accounts. We assess and discuss the elements of the framework with real-world examples and use cases, setting out research directions at the intersection of multimodal learning, multimodal analytics and computational social sciences that can benefit from our approach.

show abstract

Section: Discussionmentioning

confidence: 99%

Understanding image-text relations and news values for multimodal news analysis

Cheema

Hakimov

Müller-Budack

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Reference [11] proposed an efficient method to update building databases by using pre-disaster satellite imagery and building footprints to train a CNN, which was fine-tuned using post-disaster imagery. Reference [31] proposed a U-Net-based segmentation model to segment roads and buildings from pre-and post-disaster satellite imagery, specifically to update road networks. Progress has also been made towards real-time damage detection.…”

Section: Deep Learning For Post-disaster Damage Detectionmentioning

confidence: 99%

Post-Disaster Building Damage Detection from Earth Observation Imagery Using Unsupervised and Transferable Anomaly Detecting Generative Adversarial Networks

et al. 2020

View full text Add to dashboard Cite

We present an unsupervised deep learning approach for post-disaster building damage detection that can transfer to different typologies of damage or geographical locations. Previous advances in this direction were limited by insufficient qualitative training data. We propose to use a state-of-the-art Anomaly Detecting Generative Adversarial Network (ADGAN) because it only requires pre-event imagery of buildings in their undamaged state. This approach aids the post-disaster response phase because the model can be developed in the pre-event phase and rapidly deployed in the post-event phase. We used the xBD dataset, containing pre- and post- event satellite imagery of several disaster-types, and a custom made Unmanned Aerial Vehicle (UAV) dataset, containing post-earthquake imagery. Results showed that models trained on UAV-imagery were capable of detecting earthquake-induced damage. The best performing model for European locations obtained a recall, precision and F1-score of 0.59, 0.97 and 0.74, respectively. Models trained on satellite imagery were capable of detecting damage on the condition that the training dataset was void of vegetation and shadows. In this manner, the best performing model for (wild)fire events yielded a recall, precision and F1-score of 0.78, 0.99 and 0.87, respectively. Compared to other supervised and/or multi-epoch approaches, our results are encouraging. Moreover, in addition to image classifications, we show how contextual information can be used to create detailed damage maps without the need of a dedicated multi-task deep learning framework. Finally, we formulate practical guidelines to apply this single-epoch and unsupervised method to real-world applications.

show abstract

“…The most recent investigations into the use of deep learning models for image segmentation in satellite imagery are at an advanced stage of developments in remote sensing and earth observation [6]. The complex and ever-changing nature of satellite imagery presents a number of issues, which are now being actively addressed by researchers by actively exploring creative ways [7]. There is now work being done to improve previously developed deep learning architectures and to create new models that are specifically suited for the segmentation of satellite pictures.…”

Section: Introductionmentioning

confidence: 99%

Satellite image segmentation using Unet++ andMobileNetV2 deep learning model

Lokula,

Tirumuri,

2024

Preprint

View full text Add to dashboard Cite

Satellite image segmentation plays a pivotal role in extracting valuable information for applications like ,disaster management, environmental monitoring and land cover classification. In this work, a comprehensive approach for satellite image segmentation utilizing a fusion of UNet++ architecture and the lightweight MobileNetv2 encoder deep learning model is proposed. The UNet++ architecture, an extension of the widely adopted UNet, is employed for its ability to capture hierarchical features and enhance segmentation performance. Integrating MobileNet as the encoder provides computational efficiency, making the model well-suited for resource-constrained environments, such as satellite image analysis on edge devices. The proposed model leverages the strengths of both architectures, combining the expressive power of UNet++ with the efficiency of MobileNet. Extensive experiments are conducted on a diverse satellite image dataset, evaluating the model's segmentation accuracy, computational efficiency, and generalization capability. The results demonstrate the effectiveness of the proposed approach in achieving accurate and efficient satellite image segmentation, making it a promising solution for real-world applications in remote sensing and geospatial analysis.

show abstract

Deep learning-based aerial image segmentation with open data for disaster impact assessment

Cited by 38 publications

References 19 publications

Understanding image-text relations and news values for multimodal news analysis

Understanding image-text relations and news values for multimodal news analysis

Post-Disaster Building Damage Detection from Earth Observation Imagery Using Unsupervised and Transferable Anomaly Detecting Generative Adversarial Networks

Satellite image segmentation using Unet++ andMobileNetV2 deep learning model

Contact Info

Product

Resources

About