Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring

Abstract

Monitoring animals in their natural habitat is essential for advancement of animal behavioural studies, especially in pollination studies. Non-invasive techniques are preferred for these purposes as they reduce opportunities for research apparatus to interfere with behaviour. One potentially valuable approach is image-based tracking. However, the complexity of tracking unmarked wild animals using video is challenging in uncontrolled outdoor environments. Out-of-the-box algorithms currently present several problems in this context that can compromise accuracy, especially in cases of occlusion in a 3D environment. To address the issue, we present a novel hybrid detection and tracking algorithm to monitor unmarked insects outdoors. Our software can detect an insect, identify when a tracked insect becomes occluded from view and when it re-emerges, determine when an insect exits the camera field of view, and our software assembles a series of insect locations into a coherent trajectory. The insect detecting component of the software uses background subtraction and deep learning-based detection together to accurately and efficiently locate the insect among a cluster of wildflowers. We applied our method to track honeybees foraging outdoors using a new dataset that includes complex background detail, wind-blown foliage, and insects moving into and out of occlusion beneath leaves and among three-dimensional plant structures. We evaluated our software against human observations and previous techniques. It tracked honeybees at a rate of 86.6% on our dataset, 43% higher than the computationally more expensive, standalone deep learning model YOLOv2. We illustrate the value of our approach to quantify fine-scale foraging of honeybees. The ability to track unmarked insect pollinators in this way will help researchers better understand pollination ecology. The increased efficiency of our hybrid approach paves the way for the application of deep learning-based techniques to animal tracking in real-time using low-powered devices suitable for continuous monitoring.

Approaches, challenges and recent advances in automated bee counting devices: A review

“…There is promising evidence that the implementation of videobased systems could be successful. The recent developments of computer vision and deep learning enable monitoring of biodiversity in a fully autonomous and noninvasive way for whole seasons, which is not limited to honey bees (Høye et al, 2021;Ratnayake et al, 2021).…”

“…be of success. The recent development of computer vision and deep learning enables monitoring of biodiversity in a fully autonomous and non-invasive way for whole seasons which is not limited to honey bees(Hoye et al 2021, Ratnayake et al 2021. Such an instrument can also enhance the processing of samples in the laboratory, where automated imaging, in particular, can provide a new way of identifying and counting specimens to measure the abundance of different bee pollinators.…”

“…This could provide valuable input for developing policies that support diverse and continuous food supply in rural and agricultural landscapes. Ideally, these AI-based counters could be embedded in monitoring programs or WSNs to evaluate such policies(Ratnayake, Dyer, & Dorin, 2021). Moreover, it is also possible to detect Varroa-infested bees when a phoretic mite is attached to the bee body(Bjerge et al, 2019;Schurischuster, Remeseiro, Radeva, & Kampel, 2018).…”

Habitat fragmentation and food security in crop pollination systems

“…tracking of 'marked' pollinator individuals with remote devices; e.g. Ratnakaye et al, 2021). Finally, experimental studies provide information to calibrate biodiversity estimates in fragmented landscapes (SFAR function; Appendix S1), as well as the effect of pollinator diversity on animal-dependent crop production (Liang et al, 2016;O'Connor et al, 2017).…”

Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning

“…Application of these tracking algorithms becomes complicated in a natural setting, due to the variability of background colour/texture, solar illumination, and the increased potential for the feature of interest to be occluded between video frames (Wang et al 2019). Despite these complications, previous studies have applied tracking algorithms to wildlife videos, achieving accuracies ranging from 86.6% for honeybees (Ratnayake et al 2021) to 91.3% for elephants and humans (Bondi et al 2020).…”