Semantic Segmentation of Herbarium Specimens Using Deep Learning Techniques

Malik, Owais Ahmed; Ong, Wee-Hong; Slik, Johan Willem Frederik

doi:10.1007/978-981-15-0058-9_31

Cited by 14 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the reported results, it can be concluded that the proposed semantic segmentation-based approach for the extraction of individual intact leaves is much more efficient and accurate than the existing object detection approaches. This method has four benefits: (1) the use of the semantic segmentation model enables the extraction of individual leaves even while using a weak classifier trained on a binary image with a small dataset; (2) the semantic segmentation model used in the proposed method can be utilized as a pre-processing step for removing visual noise that exists in herbarium specimens before applying classification algorithms as used in [ 7 ] or performing feature extraction compared to object detection-based approaches; (3) the extracted leaves had a uniform white background, which could be an advantage for pre-processing tasks such as segmentation for feature extraction as shown in the result section; and (4) using the proposed method, it becomes possible to automatically extract individual leaves directly from herbarium specimen images. As opposed to the proposed method, object detection-based approaches can offer a simple solution for the location and extraction of leaves when the target task does not require precise leaf information such as phenotypic extraction of features from an individual intact leaf.…”

Section: Discussionmentioning

confidence: 99%

“…The study aimed to use the segmented information to assess three quality attributes including colorfulness, contrast, and sharpness of the images. Similarly, Hussein et al [ 7 ] proposed using deep learning semantic segmentation techniques to remove background noise in herbarium images. Adán et al [ 31 ] proposed an instance segmentation model for extracting morphological and visual information existing in herbarium specimens.…”

Section: Related Workmentioning

confidence: 99%

“…Training a classifier with the normal leaf images will present a great challenge as these catego- In this phase, we adapted DeepLabv3+ architecture, which follows the same encoderdecoder architecture. This was based on the performance of the DeepLabv3+ model on our previous work related to the segmentation of the whole herbarium specimen [7]. The model has also being widely adapted for herbarium-related studies [36].…”

Section: Phase 3: Single-leaf Classifiermentioning

confidence: 99%

“…Hence the final herbarium sheet image contains specimen with folded, overlapping, and single leaves with the addition of other non-plant objects such as color charts, barcodes, rulers, and labels. While these objects are considered useful to botanists and taxonomists, they are treated as noise when applying computer vision techniques to identify certain species [7,8]. Among the plant organs that is mostly present throughout the season, the leaf is considered important as it carries a lot of information regarding the plant [9].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation

Malik

Ong

Slik

2021

Sensors

Self Cite

View full text Add to dashboard Cite

With the increase in the digitization efforts of herbarium collections worldwide, dataset repositories such as iDigBio and GBIF now have hundreds of thousands of herbarium sheet images ready for exploration. Although this serves as a new source of plant leaves data, herbarium datasets have an inherent challenge to deal with the sheets containing other non-plant objects such as color charts, barcodes, and labels. Even for the plant part itself, a combination of different overlapping, damaged, and intact individual leaves exist together with other plant organs such as stems and fruits, which increases the complexity of leaf trait extraction and analysis. Focusing on segmentation and trait extraction on individual intact herbarium leaves, this study proposes a pipeline consisting of deep learning semantic segmentation model (DeepLabv3+), connected component analysis, and a single-leaf classifier trained on binary images to automate the extraction of an intact individual leaf with phenotypic traits. The proposed method achieved a higher F1-score for both the in-house dataset (96%) and on a publicly available herbarium dataset (93%) compared to object detection-based approaches including Faster R-CNN and YOLOv5. Furthermore, using the proposed approach, the phenotypic measurements extracted from the segmented individual leaves were closer to the ground truth measurements, which suggests the importance of the segmentation process in handling background noise. Compared to the object detection-based approaches, the proposed method showed a promising direction toward an autonomous tool for the extraction of individual leaves together with their trait data directly from herbarium specimen images.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Phase 3: Single-leaf Classifiermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation

Malik

Ong

Slik

2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…These two factors may explain why no improvements were observed with image augmentation, additional channels and subsetting models, as the model had already been well trained using the highly standardised original dataset. Modern pipelines for museum collection digitization typically follow similarly consistent standards such as uniform specimen placements, background and light environment (Hudson et al 2015;Unger et al 2016;Hussein et al 2020) suggesting that such data can be analysed with deep learning. However, high standard digitisation is time-consuming.…”

Section: Discussionmentioning

confidence: 99%

Segmenting biological specimens from photos to understand the evolution of UV plumage in passerine birds

Cooney

Varley

et al. 2021

Preprint

View full text Add to dashboard Cite

Ultraviolet (UV) colouration is thought to be an important signalling mechanism in many bird species, yet broad insights regarding the prevalence of UV plumage colouration and the factors promoting its evolution are currently lacking. Here, we develop a novel image segmentation pipeline based on deep learning that considerably outperforms classical (i.e. non-deep learning) segmentation methods, and use this to extract accurate information on whole-body plumage colouration from photographs of >24,000 museum specimens covering >4,500 species of passerine birds. Our results demonstrate that UV reflectance, particularly as a component of other colours, is widespread across the passerine radiation but is strongly phylogenetically conserved. We also find clear evidence in support of the role of light environment in promoting the evolution of UV plumage colouration, and a weak trend towards higher UV plumage reflectance among bird species with ultraviolet rather than violet-sensitive visual systems. Overall, our study provides important broad-scale insight into an enigmatic component of avian colouration, as well as demonstrating that deep learning has considerable promise for allowing new data to be bought to bare on long-standing questions in ecology and evolution.

show abstract

Analyzing trait‐climate relationships within and among taxa using machine learning and herbarium specimens

Wilde

Bragg

Cornwell

2023

American J of Botany

View full text Add to dashboard Cite

Premise Continental‐scale leaf trait studies can help explain how plants survive in different environments, but large data sets are costly to assemble at this scale. Automating the measurement of digitized herbarium collections could rapidly expand the data available to such studies. We used machine learning to identify and measure leaves from existing, digitized herbarium specimens. The process was developed, validated, and applied to analyses of relationships between leaf size and climate within and among species for two genera: Syzygium (Myrtaceae) and Ficus (Moraceae). Methods Convolutional neural network (CNN) models were used to detect and measure complete leaves in images. Predictions of a model trained with a set of 35 randomly selected images and a second model trained with 35 user‐selected images were compared using a set of 50 labeled validation images. The validated models were then applied to 1227 Syzygium and 2595 Ficus specimens digitized by the National Herbarium of New South Wales, Australia. Leaf area measurements were made for each genus and used to examine links between leaf size and climate. Results The user‐selected training method for Syzygium found more leaves (9347 vs. 8423) using fewer training masks (218 vs. 225), and found leaves with a greater range of sizes than the random image training method. Within each genus, leaf size was positively associated with temperature and rainfall, consistent with previous observations. However, within species, the associations between leaf size and environmental variables were weaker. Conclusions CNNs detected and measured leaves with levels of accuracy useful for trait extraction and analysis and illustrate the potential for machine learning of herbarium specimens to massively increase global leaf trait data sets. Within‐species relationships were weak, suggesting that population history and gene flow have a strong effect at this level. Herbarium specimens and machine learning could expand sampling of trait data within many species, offering new insights into trait evolution.

show abstract

Semantic Segmentation of Herbarium Specimens Using Deep Learning Techniques

Cited by 14 publications

References 10 publications

Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation

Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation

Segmenting biological specimens from photos to understand the evolution of UV plumage in passerine birds

Analyzing trait‐climate relationships within and among taxa using machine learning and herbarium specimens

Contact Info

Product

Resources

About