A deep active learning system for species identification and counting in camera trap images

Abstract

A typical camera trap survey may produce millions of images that require slow, expensive manual review. Consequently, critical conservation questions may be answered too slowly to support decision‐making. Recent studies demonstrated the potential for computer vision to dramatically increase efficiency in image‐based biodiversity surveys; however, the literature has focused on projects with a large set of labelled training images, and hence many projects with a smaller set of labelled images cannot benefit from existing machine learning techniques. Furthermore, even sizable projects have struggled to adopt computer vision methods because classification models overfit to specific image backgrounds (i.e. camera locations). In this paper, we combine the power of machine intelligence and human intelligence via a novel active learning system to minimize the manual work required to train a computer vision model. Furthermore, we utilize object detection models and transfer learning to prevent overfitting to camera locations. To our knowledge, this is the first work to apply an active learning approach to camera trap images. Our proposed scheme can match state‐of‐the‐art accuracy on a 3.2 million image dataset with as few as 14,100 manual labels, which means decreasing manual labelling effort by over 99.5%. Our trained models are also less dependent on background pixels, since they operate only on cropped regions around animals. The proposed active deep learning scheme can significantly reduce the manual labour required to extract information from camera trap images. Automation of information extraction will not only benefit existing camera trap projects, but can also catalyse the deployment of larger camera trap arrays.

Revisiting animal photo‐identification using deep metric learning and network analysis

“…Using accurate segmentation of animal body (Brodrick et al., 2019; He et al., 2017) will undoubtedly be a solution against side effects of rectangular cropping. Moreover, this pipeline can be used in an active learning strategy where the machine learning model is assisted by human intervention on some specific cases (Norouzzadeh et al., 2021). Indeed, using the proposed distance threshold in the Euclidean space, one can iteratively enrich the training dataset after manual checking of the most confident Top‐1 candidates (below a small distance threshold, to guarantee optimal TN rate) and re‐run the estimation procedure.…”

“…A range of CNN‐based tools are now available for object detection and already used for animal detection (Parham et al., 2018; Sadegh Norouzzadeh et al., 2019; Schneider et al., 2018). Among other options including YOLO (Bochkovskiy et al., 2020; Redmon et al., 2016) and Mask R‐CNN (He et al., 2017), RetinaNet (Lin et al., 2017) is a CNN‐based object detector able to detect a series of predefined object classes (e.g.…”

Robust ecological analysis of camera trap data labelled by a machine learning model

“…This resulted in long sequences of very similar images, for example showing an animal walking in front of the camera (Figure S1). (Norouzzadeh et al, 2019). This approach posed a challenge for maintaining class balances in the training and validation sets, but it reduced the risk of non-independent training and validation sets.…”

Revisiting animal photo-identification using deep metric learning and network analysis

“…Using accurate segmentation of animal body (He et al ., 2017; Brodrick et al ., 2019) will undoubtedly be a solution against side effects of rectangular cropping. Moreover, this pipeline can be used in an active learning strategy where the machine learning model is assisted by human Norouzzadeh et al . (2021).…”

“…A range of CNN-based tools are now available for object detection, already used for animal detection (Parham et al ., 2018; Schneider et al ., 2018; Sadegh Norouzzadeh et al ., 2019). Among other options including YOLO (Redmon et al ., 2016; Bochkovskiy et al ., 2020) and Mask R-CNN (He et al ., 2017), RetinaNet (Lin et al ., 2017) is a CNN-based object detector able to detect a series of predefined object classes ( e .…”