Combining Human Computing and Machine Learning to Make Sense of Big (Aerial) Data for Disaster Response

Abstract

Aerial imagery captured via unmanned aerial vehicles (UAVs) is playing an increasingly important role in disaster response. Unlike satellite imagery, aerial imagery can be captured and processed within hours rather than days. In addition, the spatial resolution of aerial imagery is an order of magnitude higher than the imagery produced by the most sophisticated commercial satellites today. Both the United States Federal Emergency Management Agency (FEMA) and the European Commission's Joint Research Center ( JRC) have noted that aerial imagery will inevitably present a big data challenge. The purpose of this article is to get ahead of this future challenge by proposing a hybrid crowdsourcing and real-time machine learning solution to rapidly process large volumes of aerial data for disaster response in a time-sensitive manner. Crowdsourcing can be used to annotate features of interest in aerial images (such as damaged shelters and roads blocked by debris). These human-annotated features can then be used to train a supervised machine learning system to learn to recognize such features in new unseen images. In this article, we describe how this hybrid solution for image analysis can be implemented as a module (i.e., Aerial Clicker) to extend an existing platform called Artificial Intelligence for Disaster Response (AIDR), which has already been deployed to classify microblog messages during disasters using its Text Clicker module and in response to Cyclone Pam, a category 5 cyclone that devastated Vanuatu in March 2015. The hybrid solution we present can be applied to both aerial and satellite imagery and has applications beyond disaster response such as wildlife protection, human rights, and archeological exploration. As a proof of concept, we recently piloted this solution using very high-resolution aerial photographs of a wildlife reserve in Namibia to support rangers with their wildlife conservation efforts (SAVMAP project, http://lasig.epfl.ch/savmap). The results suggest that the platform we have developed to combine crowdsourcing and machine learning to make sense of large volumes of aerial images can be used for disaster response.

Detecting mammals in UAV images: Best practices to address a substantially imbalanced dataset with deep learning

“…An initial ground truth (convex hull polygons of large animals) was provided by MicroMappers 4 (Ofli et al, 2016) and consisted of 976 annotations in a subset of the 654 images, which was completed within three days. Since they were based on crowdsourced volunteers efforts, some of the annotations were coarse or erroneous in that they occasionally omitted an animal, were not very accurate position-wise, or included multiple individuals in one annotation at once ( Figure 1a).…”

Seguridad metropolitana mediante el uso coordinado de Drones

“…Con el avance progresivo de las cámaras de video [28], se convierten literalmente en los ojos de las fuerzas policiales en el aire, en la que permite efectuar la medición de variables como aceleración y velocidad del dron respecto a la superficie de navegación, por lo cual, el registro de un evento estático o en movimiento es extrapolable.…”

Detecting animals in African Savanna with UAVs and the crowds

“…In this paper, we present a data-driven machine learning system for the semi-automatic detection of large mammals in the Savanna ecosystem characterized by complex land-cover. We perform animal detection on a set of sub-decimeter resolution images acquired in the Namibian Kalahari desert and train our system using animals annotated by digital volunteers using the Micromappers crowdsourcing platform (Ofli et al, 2016). We focused on large mammals for two main reasons: first, they stood out compared to the background, while smaller animals such as meerkats are not clearly visible and could be too easily confused with rocks or bushes by the volunteers.…”