Automatic detection and classification of honey bee comb cells using deep learning

Abstract

In a scenario of worldwide honey bee decline, assessing colony strength is becoming increasingly important for sustainable beekeeping. Temporal counts of number of comb cells with brood and food reserves offers researchers data for multiple applications, such as modelling colony dynamics, and beekeepers information on colony strength, an indicator of colony health and honey yield. Counting cells manually in comb images is labour intensive, tedious, and prone to error. Herein, we developed a free software, named DeepBee©, capable of automatically detecting cells in comb images and classifying their contents into seven classes. By distinguishing cells occupied by eggs, larvae, capped brood, pollen, nectar, honey, and other, DeepBee© allows an unprecedented level of accuracy in cell classification. Using Circle Hough Transform and the semantic segmentation technique, we obtained a cell detection rate of 98.7%, which is 16.2% higher than the best result found in the literature. For classification of comb cells, we trained and evaluated thirteen different convolutional neural network (CNN) architectures, including: DenseNet (121, 169 and 201); InceptionResNetV2; InceptionV3; MobileNet; MobileNetV2; NasNet; NasNetMobile; ResNet50; VGG (16 and 19) and Xception. MobileNet revealed to be the best compromise between training cost, with~9 s for processing all cells in a comb image, and accuracy, with an F1-Score of 94.3%. We show the technical details to build a complete pipeline for classifying and counting comb cells and we made the CNN models, source code, and datasets publicly available. With this effort, we hope to have expanded the frontier of apicultural precision analysis by providing a tool with high performance and source codes to foster improvement by third parties (https://github.com/AvsThiago/DeepBeesource).

Markerless tracking of an entire honey bee colony

“…2 . Previous work 57 has proposed a similar approach to the detection and recognition of all cell types (brood, honey, nectar, pollen, larva, egg), however, in a standardized imaging setup and based on empty frames that were removed from the hive. While our imaging arrangement and the noise introduced by the background extraction algorithm currently do not provide the detail needed to detect eggs or pollen, our approach does provide access to continuous measurements of the brood counts in a living colony.…”

Automated image analysis to assess hygienic behaviour of honeybees

“…Further, to validate our model against a publicly available dataset obtained with a different equipment and under different acquisition conditions, 100 images were randomly selected among those used in [ 18 ] (available at https://github.com/AvsThiago/DeepBee-source/archive/release-0.1.zip ), and we compared the results obtained when processing them to those obtained on the images in I_{Test} , which were more cautiously cropped by experts to strictly include the comb area.…”

“…HoneyBeeComplete displays the classification of capped brood cells with a detection rate of 97.4% [ 14 ]; its promising results motivate their usage during subsequent studies [ 15 ]; HiveAnalyzer shows the ability to classify other cells in addition to capped brood through linear Support Vector Machines (SVM) with a classification rate of 94% [ 16 ]; CombCount displays the detection of both capped brood and capped honey although a user is required to discriminate between the two with selection tools [ 17 ]. Recently, a completely automatic tool using convolutional neural networks (CNNs), DeepBee, showed the classification of seven different comb cell classes with a detection rate of 98.7% [ 18 ].…”

Research project on field data collection for honey bee colony model evaluation

“…The population assessment of adult bees was done by weighting all frames, supers and hive bottoms with and without bees. The number of bees per frame was estimated from the total weight of bees, Brood and provisions were assessed by photographing combs and using a newly developed method, automatic detection and classification of honey bee comb cells using deep learning (Alves et al, 2020). During adult population assessment, every comb of the colony was inspected.…”

“…During adult population assessment, every comb of the colony was inspected. Except for pure honey combs and empty combs, both sides of all combs were photographed with a digital camera inside a wooden tunnel with a built-in LED lighting, as in Alves et al (2020). During the preparatory phase of the project, the set-up, which was originally developed under Portuguese conditions and for Langstroth frame measures, was adapted for Danish conditions and the Norwegian frame measures used in the experimental hives.…”