Scene‐specific convolutional neural networks for video‐based biodiversity detection

Abstract

Finding, counting and identifying animals is a central challenge in ecology. Most studies are limited by the time and cost of fieldwork by human observers. To increase the spatial and temporal breadth of sampling, ecologists are adopting passive image‐based monitoring approaches. While passive monitoring can expand data collection, a remaining obstacle is finding the small proportion of images containing ecological objects among the majority of frames containing only background scenes. I proposed a scene‐specific convolutional neural network for detecting animals of interest within long duration time‐lapse videos. Convolutional neural networks are a type of deep learning algorithm that have recently made significant advances in image classification. The approach was tested on videos of floral visitation by hummingbirds. Despite low frame rates, poor image quality, and complex video conditions, the model correctly classified over 90% of frames containing hummingbirds. Combining motion detection and image classification can substantially reduce the time investment in scoring images from passive monitoring studies. These results underscore the promise of deep learning to lead ecology into greater automation using passive image analysis. To help facilitate future applications, I created a desktop executable that can be used to apply pre‐trained models to videos, as well as reproducible scripts for training new models on local and cloud environments.

Monitoring of Coral Reefs Using Artificial Intelligence: A Feasible and Cost-Effective Approach

“…This study builds on previous research highlighting the benefits of machine learning in automated analyses of underwater coral reef images [17,18]. While machine learning can vastly accelerate the rate at which images are analysed for ecological studies, more advanced techniques can render more useful applications in ecology by reducing the error introduced by automated classification (e.g., [15]). Here, we explored the applications deep learning as a tool to assist coral reef monitoring and evaluated its performance on automated image annotation.…”

“…One of the greatest advances of deep learning is that it makes it possible to automatically discover the features needed for classification, and thus is capable of resolving intricate structures in high-dimensional data [12]. As such, deep learning has set new standards in image [13] and speech [14] recognition, as well as contributing to advances in drug discovery, brain circuit reconstruction [12], ecology [15] and remote sensing [16]. Here, we pose the central question of whether advances in automated image recognition could accelerate image analysis in coral reef monitoring and at what cost.…”

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

“…15 show that this method outperforms T2-FMOG [8], SuBSENSE [178], and DECOLOR [233]. For biodiversity detection in terrestrial and marine environments, Weinstein [206] employed the GoogLeNet architecture integrated in a software called DeepMeerkat 16 . Experiments on humming bird videos show robust performance in challenging outdoor scenes where moving foliages occur.…”

ClassifyMe: A Field-Scouting Software for the Identification of Wildlife in Camera Trap Images

“…Motion Meerkat is a software application which also utilises computer vision in the form of mixture of Gaussian models to detect motion in videos which reduces the number of hours required for researcher review [62]. DeepMeerkat provides similar functionality using convolutional neural networks to monitor for the presence of specific objects (e.g., hummingbirds) in videos [63]. There is a further, wide range of software available including Renamer [64] and VIXEN [65] to support camera trap data management.…”