Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation

Abstract

With the increase in the digitization efforts of herbarium collections worldwide, dataset repositories such as iDigBio and GBIF now have hundreds of thousands of herbarium sheet images ready for exploration. Although this serves as a new source of plant leaves data, herbarium datasets have an inherent challenge to deal with the sheets containing other non-plant objects such as color charts, barcodes, and labels. Even for the plant part itself, a combination of different overlapping, damaged, and intact individual leaves exist together with other plant organs such as stems and fruits, which increases the complexity of leaf trait extraction and analysis. Focusing on segmentation and trait extraction on individual intact herbarium leaves, this study proposes a pipeline consisting of deep learning semantic segmentation model (DeepLabv3+), connected component analysis, and a single-leaf classifier trained on binary images to automate the extraction of an intact individual leaf with phenotypic traits. The proposed method achieved a higher F1-score for both the in-house dataset (96%) and on a publicly available herbarium dataset (93%) compared to object detection-based approaches including Faster R-CNN and YOLOv5. Furthermore, using the proposed approach, the phenotypic measurements extracted from the segmented individual leaves were closer to the ground truth measurements, which suggests the importance of the segmentation process in handling background noise. Compared to the object detection-based approaches, the proposed method showed a promising direction toward an autonomous tool for the extraction of individual leaves together with their trait data directly from herbarium specimen images.

[Retracted] Deep Learning‐Based Leaf Region Segmentation Using High‐Resolution Super HAD CCD and ISOCELL GW1 Sensors

“…The leaf region segmentation is challenging when the plant images have overlapping/occluded leaves and complex backgrounds. Most presently available leaf segmentation methods [12,15,17,24,26,36,37] were designed to work specifically with certain acquisition circumstances. As a consequence, these techniques are not able to give good results in field conditions.…”

An image dataset of cleared, x-rayed, and fossil leaves vetted to plant family for human and machine learning

“…However, studying leaf comparative morphology is not simple because leaves only represent part of the visual field of a herbarium sheet and appear, with overlaps, at many different angles and sizes. Computer-vision algorithms that blur text or segment leaves from background or from other plant material are likely to help solve this issue (Hussein et al 2020(Hussein et al , 2021Weaver et al 2020;de Lutio et al 2021). However, many visual distractors remain, and critical details of higherorder venation are often not visible in digitized herbarium sheets.…”

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

“…Exceptional masks are produced for a wide variety of leaf shapes, even for lobed and toothed taxa. LeafMachine2 successfully ignores mounting tape and returns complete leaf masks, bypassing the need for shape matching or connected component analyses as is required by other methods (Hussein et al, 2021a ). Accurate segmentation of individual leaves is possible from a group of leaves, even when obstructions are present (Figure 2 , leaves E, K–Q).…”