Image-based classification of plant genus and family for trained and untrained plant species

Abstract

BackgroundModern plant taxonomy reflects phylogenetic relationships among taxa based on proposed morphological and genetic similarities. However, taxonomical relation is not necessarily reflected by close overall resemblance, but rather by commonality of very specific morphological characters or similarity on the molecular level. It is an open research question to which extent phylogenetic relations within higher taxonomic levels such as genera and families are reflected by shared visual characters of the constituting species. As a consequence, it is even more questionable whether the taxonomy of plants at these levels can be identified from images using machine learning techniques.ResultsWhereas previous studies on automated plant identification from images focused on the species level, we investigated classification at higher taxonomic levels such as genera and families. We used images of 1000 plant species that are representative for the flora of Western Europe. We tested how accurate a visual representation of genera and families can be learned from images of their species in order to identify the taxonomy of species included in and excluded from learning. Using natural images with random content, roughly 500 images per species are required for accurate classification. The classification accuracy for 1000 species amounts to 82.2% and increases to 85.9% and 88.4% on genus and family level. Classifying species excluded from training, the accuracy significantly reduces to 38.3% and 38.7% on genus and family level. Excluded species of well represented genera and families can be classified with 67.8% and 52.8% accuracy.ConclusionOur results show that shared visual characters are indeed present at higher taxonomic levels. Most dominantly they are preserved in flowers and leaves, and enable state-of-the-art classification algorithms to learn accurate visual representations of plant genera and families. Given a sufficient amount and composition of training data, we show that this allows for high classification accuracy increasing with the taxonomic level and even facilitating the taxonomic identification of species excluded from the training process.Electronic supplementary materialThe online version of this article (10.1186/s12859-018-2474-x) contains supplementary material, which is available to authorized users.

Flowers, leaves or both? How to obtain suitable images for automated plant identification

“…However, such images inhibit a wide range of quality. A widely known example is the PlantCLEF dataset [13], which is used as benchmark for various computer vision tasks [14–18]. In this collection, each image is assigned a posteriori to one of seven categories (entire, leaf, leaf scan, flower, fruit, stem and branch).…”

Analysis of Pre-Trained Convolutional Neural Networks to Build a Flower Classification System

“…Most of the existing systems are using leaves to identify plants. This can be problematic as discussed previously in this paper, the major reason being the high intraclass variations and inter-class similarities [1]. Another reason for leaves not being the most accurate way of identifying plants is that the climatic conditions and diseases that play a great role in the appearance of leaves.…”

“…Existing systems using KNN, MLNN, BPNN, SVM, etc. that classify plants on the basis of their leaf shape, skeleton, color, texture do not give a desirable accuracy for precise classification [1]. Due to similar physical structures, leaves of different species tend to be often confused as the same and leaves from the same species may look different altogether.…”

“…There are innumerable plant species and each plant has a variety of flowers that differ in their color and texture. Therefore, flower classification is a complex task and requires adequate knowledge and research [1,2]. Nature enthusiasts and tourists are attracted to places of natural and cultural diversity.…”

Flora Capture: a citizen science application for collecting structured plant observations

“…Deep CNNs have been demonstrated to facilitate classification accuracies that are on par with human performance for general object recognition tasks [ 8 ] as well as for fine-grained species identification tasks [ 11 ]. Latest studies on automated image-based plant identification show identification accuracy that at least reaches humans’ identification abilities for common plants [ 1 , 7 , 12 ]. However, with more than 380,000 described species worldwide [ 13 ], automated plant identification still constitutes a challenging image recognition problem, further complicated by low interspecific variability and high intraspecific variability for many species (cp.…”