Deep leaf: Mask R-CNN based leaf detection and segmentation from digitized herbarium specimen images

Triki, Abdelaziz; Bouaziz, Bassem; Gaikwad, Jitendra; Mahdi, Walid

doi:10.1016/j.patrec.2021.07.003

Cited by 31 publications

(17 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These frameworks are extremely flexible, well‐supported, and surprisingly approachable. As a result, many recent projects have also coalesced around these two frameworks with great success, including efforts to segment leaves (Younis et al, 2020; Triki et al, 2020, 2021; Guo et al, 2021; Hussein et al, 2021b; Gu et al, 2022; Ott and Lautenschlager, 2022), segment plant tissue (Love et al, 2021; Goëau et al, 2022; Milleville et al, 2023), isolate plant organs (Davis et al, 2020; Pearson et al, 2020; Triki et al, 2020; Ott and Lautenschlager, 2022), extract specimen label data (Milleville et al, 2023), isolate diseased or damaged leaf tissue (Kaur et al, 2022; Mu et al, 2022; Kavitha Lakshmi and Savarimuthu, 2023), measure bird skeletons (Weeks et al, 2023), isolate preserved snakes (Curlis et al, 2022), segment fossils (Panigrahi et al, 2022), or remotely monitor phenology (Mann et al, 2022). However, rather than relying on a single machine learning architecture to extract trait and archival data from specimens, we developed a modular framework of seven different machine learning algorithms that work in tandem to comprehensively process each image (Table 2, Figure 1).…”

Section: Term Definitionmentioning

confidence: 99%

“…To overcome these limitations, many groups turned to machine learning algorithms, typically some kind of convolutional neural network (CNN), which can categorize individual pixels as members of discrete classes (Ott et al, 2020; Weaver et al, 2020; Younis et al, 2020; Triki et al, 2020, 2021; Goëau et al, 2020, 2022; Guo et al, 2021; Love et al, 2021; Hussein et al, 2021b; Gu et al, 2022; Ott and Lautenschlager, 2022; Milleville et al, 2023). For the task of isolating and measuring individual leaves, semantic segmentation algorithms still lack the power to resolve complex situations (e.g., overlapping leaves) because they produce one mask that contains all leaf pixels and require postprocessing to obtain usable results (Weaver et al, 2020; Hussein et al, 2021b, 2022).…”

Section: Term Definitionmentioning

confidence: 99%

See 1 more Smart Citation

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

Weaver,

Smith

2023

Appl Plant Sci

View full text Add to dashboard Cite

PremiseQuantitative plant traits play a crucial role in biological research. However, traditional methods for measuring plant morphology are time consuming and have limited scalability. We present LeafMachine2, a suite of modular machine learning and computer vision tools that can automatically extract a base set of leaf traits from digital plant data sets.MethodsLeafMachine2 was trained on 494,766 manually prepared annotations from 5648 herbarium images obtained from 288 institutions and representing 2663 species; it employs a set of plant component detection and segmentation algorithms to isolate individual leaves, petioles, fruits, flowers, wood samples, buds, and roots. Our landmarking network automatically identifies and measures nine pseudo‐landmarks that occur on most broadleaf taxa. Text labels and barcodes are automatically identified by an archival component detector and are prepared for optical character recognition methods or natural language processing algorithms.ResultsLeafMachine2 can extract trait data from at least 245 angiosperm families and calculate pixel‐to‐metric conversion factors for 26 commonly used ruler types.DiscussionLeafMachine2 is a highly efficient tool for generating large quantities of plant trait data, even from occluded or overlapping leaves, field images, and non‐archival data sets. Our project, along with similar initiatives, has made significant progress in removing the bottleneck in plant trait data acquisition from herbarium specimens and shifted the focus toward the crucial task of data revision and quality control.

show abstract

Section: Term Definitionmentioning

confidence: 99%

Section: Term Definitionmentioning

confidence: 99%

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

Weaver,

Smith

2023

Appl Plant Sci

View full text Add to dashboard Cite

show abstract

“…Geetharamani and Arun Pandian proposed a nine-layer DCNN for disease identification [ 9 ]. In [ 10 ], Triki et al proposed a leaf detection and segmentation model, deep leaf, which was based on Mask-RCNN and used morphological characteristics in plant specimens. Liu et al applied a long short-term memory network-based variational autoencoder to extract the sequential feature of the application running time [ 11 ].…”

Section: Related Workmentioning

confidence: 99%

Improved AlexNet with Inception-V4 for Plant Disease Diagnosis

Deng

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Timely disease detection and pest treatment are key issues in modern agricultural production, especially in large-scale crop agriculture. However, it is very time and effort-consuming to identify plant diseases manually. This paper proposes a deep learning model for agricultural crop disease identification based on AlexNet and Inception-V4. AlexNet and Inception-V4 are combined and modified to achieve an efficient but good performance. Experimental results on the expanded PlantVillage dataset show that the proposed model outperforms the compared methods: AlexNet, VGG11, Zenit, and VGG16, in terms of accuracy and F1 scores. The proposed model obtains the highest accuracy for corn, tomato, grape, and apple: 94.5%, 94.8%, 92.3%, and 96.5%, respectively. Also, the highest F1 scores for corn, tomato, grape, and apple: 0.938, 0.910, 0.945, and 0.924, respectively, are obtained. The results indicate that the proposed method has promising generalization ability in crop disease identification.

show abstract

“…We used predictions from the model to examine associations between leaf size and climate. This study differs from previous studies (Weaver et al, 2020;Triki et al, 2021) that used machine learning to measure leaves because we used a minimal training approach. Such an approach can be used to create large amounts of data and increase our understanding of relationships between leaf traits and climate at different taxonomic levels.…”

mentioning

confidence: 94%

Analyzing trait‐climate relationships within and among taxa using machine learning and herbarium specimens

Wilde

Bragg

Cornwell

2023

American J of Botany

View full text Add to dashboard Cite

Premise Continental‐scale leaf trait studies can help explain how plants survive in different environments, but large data sets are costly to assemble at this scale. Automating the measurement of digitized herbarium collections could rapidly expand the data available to such studies. We used machine learning to identify and measure leaves from existing, digitized herbarium specimens. The process was developed, validated, and applied to analyses of relationships between leaf size and climate within and among species for two genera: Syzygium (Myrtaceae) and Ficus (Moraceae). Methods Convolutional neural network (CNN) models were used to detect and measure complete leaves in images. Predictions of a model trained with a set of 35 randomly selected images and a second model trained with 35 user‐selected images were compared using a set of 50 labeled validation images. The validated models were then applied to 1227 Syzygium and 2595 Ficus specimens digitized by the National Herbarium of New South Wales, Australia. Leaf area measurements were made for each genus and used to examine links between leaf size and climate. Results The user‐selected training method for Syzygium found more leaves (9347 vs. 8423) using fewer training masks (218 vs. 225), and found leaves with a greater range of sizes than the random image training method. Within each genus, leaf size was positively associated with temperature and rainfall, consistent with previous observations. However, within species, the associations between leaf size and environmental variables were weaker. Conclusions CNNs detected and measured leaves with levels of accuracy useful for trait extraction and analysis and illustrate the potential for machine learning of herbarium specimens to massively increase global leaf trait data sets. Within‐species relationships were weak, suggesting that population history and gene flow have a strong effect at this level. Herbarium specimens and machine learning could expand sampling of trait data within many species, offering new insights into trait evolution.

show abstract

Deep leaf: Mask R-CNN based leaf detection and segmentation from digitized herbarium specimen images

Cited by 31 publications

References 8 publications

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

Improved AlexNet with Inception-V4 for Plant Disease Diagnosis

Analyzing trait‐climate relationships within and among taxa using machine learning and herbarium specimens

Contact Info

Product

Resources

About