Detecting mammals in UAV images: Best practices to address a substantially imbalanced dataset with deep learning

Kellenberger, Benjamin; Marcos, Diego; Tuia, Devis

doi:10.1016/j.rse.2018.06.028

Cited by 252 publications

(189 citation statements)

References 37 publications

Supporting

Mentioning

163

Contrasting

Order By: Relevance

“…Our animal detection algorithm performed better than previously published algorithms for aerial imagery of mammals in similar habitats (Kellenberger et al, ; Rey et al, ; Sirmacek et al, ), whilst also being able to differentiate between animal species. Furthermore, this is likely the first evidence of an algorithm detecting animals that multiple layers of humans were not able to detect.…”

Section: Discussionmentioning

confidence: 63%

“…To simultaneously increase the sampling efficiency and standardize the animal detection system, Unmanned Aerial Vehicles (UAVs) or microlight aircrafts with cameras and an automated image object detection algorithm are considered an alternative to manned aircrafts with human observers (Colefax, Butcher, & Kelaher, 2018;Linchant, Lisein, Semeki, Lejeune, & Vermeulen, 2015;Rey, Volpi, Joost, & Tuia, 2017;Sirmacek et al, 2012). In the past decade, this proposed method has been explored and tested in various studies (Hodgson et al, 2018;Kellenberger, Marcos, & Tuia, 2018;Rey et outperform humans in detecting animals from the air when supplied with images taken at a fixed rate. computer vision, convolutional neural network, deep machine learning, drones, game census, image recognition, savanna, wildlife survey Van Gemert et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving the precision and accuracy of animal population estimates with aerial image object detection

et al. 2019

View full text Add to dashboard Cite

Animal population sizes are often estimated using aerial sample counts by human observers, both for wildlife and livestock. The associated methods of counting remained more or less the same since the 1970s, but suffer from low precision and low accuracy of population estimates. Aerial counts using cost‐efficient Unmanned Aerial Vehicles or microlight aircrafts with cameras and an automated animal detection algorithm can potentially improve this precision and accuracy. Therefore, we evaluated the performance of the multi‐class convolutional neural network RetinaNet in detecting elephants, giraffes and zebras in aerial images from two Kenyan animal counts. The algorithm detected 95% of the number of elephants, 91% of giraffes and 90% of zebras that were found by four layers of human annotation, of which it correctly detected an extra 2.8% of elephants, 3.8% giraffes and 4.0% zebras that were missed by all humans, while detecting only 1.6 to 5.0 false positives per true positive. Furthermore, the animal detections by the algorithm were less sensitive to the sighting distance than humans were. With such a high recall and precision, we posit it is feasible to replace manual aerial animal count methods (from images and/or directly) by only the manual identification of image bounding boxes selected by the algorithm and then use a correction factor equal to the inverse of the undercounting bias in the calculation of the population estimates. This correction factor causes the standard error of the population estimate to increase slightly compared to a manual method, but this increase can be compensated for when the sampling effort would increase by 23%. However, an increase in sampling effort of 160% to 1,050% can be attained with the same expenses for equipment and personnel using our proposed semi‐automatic method compared to a manual method. Therefore, we conclude that our proposed aerial count method will improve the accuracy of population estimates and will decrease the standard error of population estimates by 31% to 67%. Most importantly, this animal detection algorithm has the potential to outperform humans in detecting animals from the air when supplied with images taken at a fixed rate.

show abstract

Section: Discussionmentioning

confidence: 63%

Section: Introductionmentioning

confidence: 99%

Improving the precision and accuracy of animal population estimates with aerial image object detection

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The significant difference between deep learning and classic visual recognition methods is that deep learning methods automatically learn hierarchical features from a huge amount of data rather than requiring the engineering of features by hand. Convolutional neural networks (CNNs), a recent family of deep learning algorithms, normally have more than one hidden layer; thus, they are able to extract more useful feature representations from a large number of input images for object detection and have been used in detecting large mammals in large datasets with a higher accuracy than those of traditional machine learning algorithms, such as EESVM (80% correct detections for a precision of 30% [100] vs. 75% correct detections for a precision of 10% [3]). However, extremely large training datasets are needed when applying deep learning, and it is a black-box solution; consequently, the trained models are unexplainable [99].…”

Section: Deep Learningmentioning

confidence: 99%

Surveying Wild Animals from Satellites, Manned Aircraft and Unmanned Aerial Systems (UASs): A Review

2019

View full text Add to dashboard Cite

This article reviews studies regarding wild animal surveys based on multiple platforms, including satellites, manned aircraft, and unmanned aircraft systems (UASs), and focuses on the data used, animal detection methods, and their accuracies. We also discuss the advantages and limitations of each type of remote sensing data and highlight some new research opportunities and challenges. Submeter very-high-resolution (VHR) spaceborne imagery has potential in modeling the population dynamics of large (>0.6 m) wild animals at large spatial and temporal scales, but has difficulty discerning small (<0.6 m) animals at the species level, although high-resolution commercial satellites, such as WorldView-3 and -4, have been able to collect images with a ground resolution of up to 0.31 m in panchromatic mode. This situation will not change unless the satellite image resolution is greatly improved in the future. Manned aerial surveys have long been employed to capture the centimeter-scale images required for animal censuses over large areas. However, such aerial surveys are costly to implement in small areas and can cause significant disturbances to wild animals because of their noise. In contrast, UAS surveys are seen as a safe, convenient and less expensive alternative to ground-based and conventional manned aerial surveys, but most UASs can cover only small areas. The proposed use of UAS imagery in combination with VHR satellite imagery would produce critical population data for large wild animal species and colonies over large areas. The development of software systems for automatically producing image mosaics and recognizing wild animals will further improve survey efficiency.

show abstract

“…Also, we disable curriculum learning and train Fig. 7: Basic architecture for our animal detector, following [8]. We employ the main blocks of a ResNet-18 pretrained on ImageNet and add two more MLPs and ReLU nonlinearities.…”

Section: B Model Setupmentioning

confidence: 99%

Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning

Kellenberger

Marcos

Lobry

et al. 2019

IEEE Trans. Geosci. Remote Sensing

Self Cite

106

View full text Add to dashboard Cite

We present an Active Learning (AL) strategy for re-using a deep Convolutional Neural Network (CNN)-based object detector on a new dataset. This is of particular interest for wildlife conservation: given a set of images acquired with an Unmanned Aerial Vehicle (UAV) and manually labeled gound truth, our goal is to train an animal detector that can be re-used for repeated acquisitions, e.g. in follow-up years. Domain shifts between datasets typically prevent such a direct model application. We thus propose to bridge this gap using AL and introduce a new criterion called Transfer Sampling (TS). TS uses Optimal Transport to find corresponding regions between the source and the target datasets in the space of CNN activations. The CNN scores in the source dataset are used to rank the samples according to their likelihood of being animals, and this ranking is transferred to the target dataset. Unlike conventional AL criteria that exploit model uncertainty, TS focuses on very confident samples, thus allowing a quick retrieval of true positives in the target dataset, where positives are typically extremely rare and difficult to find by visual inspection. We extend TS with a new window cropping strategy that further accelerates sample retrieval. Our experiments show that with both strategies combined, less than half a percent of oracle-provided labels are enough to find almost 80% of the animals in challenging sets of UAV images, beating all baselines by a margin.

show abstract

Detecting mammals in UAV images: Best practices to address a substantially imbalanced dataset with deep learning

Cited by 252 publications

References 37 publications

Improving the precision and accuracy of animal population estimates with aerial image object detection

Improving the precision and accuracy of animal population estimates with aerial image object detection

Surveying Wild Animals from Satellites, Manned Aircraft and Unmanned Aerial Systems (UASs): A Review

Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning

Contact Info

Product

Resources

About