Machine Learning Using Digitized Herbarium Specimens to Advance Phenological Research

Abstract

<jats:title>Abstract</jats:title> <jats:p>Machine learning (ML) has great potential to drive scientific discovery by harvesting data from images of herbarium specimens—preserved plant material curated in natural history collections—but ML techniques have only recently been applied to this rich resource. ML has particularly strong prospects for the study of plant phenological events such as growth and reproduction. As a major indicator of climate change, driver of ecological processes, and critical determinant of plant fitness, plant phenology is an important frontier for the application of ML techniques for science and society. In the present article, we describe a generalized, modular ML workflow for extracting phenological data from images of herbarium specimens, and we discuss the advantages, limitations, and potential future improvements of this workflow. Strategic research and investment in specimen-based ML methods, along with the aggregation of herbarium specimen data, may give rise to a better understanding of life on Earth.</jats:p>

Plants meet machines: Prospects in machine learning for plant biology

“…They are also being applied to questions of comparative genomics (e.g., Xu and Jackson, 2019) and gene expression (Mochida et al, 2018) and to conduct high-throughput phenotyping (e.g., Singh et al, 2016;Ubbens and Stavness, 2017) for agricultural and ecological research. Moreover, novel approaches are poised to revolutionize studies of plant phenology (e.g., Pearson et al, 2020) and functional traits through application to more than 30 million images of herbarium specimens now available at iDigBio (http://www.idigb io.org) as well as other digital repositories.…”

“…These results are very promising for extracting a broad range of accurate annotations in a fully automated way. Such approaches are also being applied to identification of plant phenophase (i.e., bud, flower, fruit), which is important for assessing the effects of climate change on plant growth and reproduction and for comparing plant responses with those of pollinators, migratory birds, and other species that rely on plants for food and/or nesting sites (see, e.g., Lorieul et al, 2019;Pearson et al, 2020;Brenskelle et al, 2020;Goëau et al, 2020). Likewise, other evolutionary or ecological traits, such as leaf shape and size, leaf margins, and flower color, could also potentially be scored from images of herbarium specimens.…”

“…A major hurdle, however, is that to harness the full power of herbarium specimens for phenological research requires counting reproductive structures, which can be time consuming. Thus, automated recognition of reproductive structures on herbarium specimens is a key goal in current phenological research (Lorieul et al, 2019;Pearson et al, 2020). Two papers in this special issue address plant reproductive phenology.…”

Landscape Analysis for the Specimen Data Refinery

“…Taxize is an R package developed specifically for this purpose (Chamberlain and Szöcs 2013) as well as Taxosaurus, a thesaurus for taxonomic names, along with a number of other resources. It has been demonstrated that simple processing of taxon names can considerably increase the matching of names from different sources (Patterson et al 2016).…”

The Newspaper Navigator Dataset

“…Our work is a case study in partnering machine learning projects with volunteer crowdsourcing initiatives, a promising paradigm in which annotators are volunteers who learn about a new topic by participating. With the growing efforts of cultural heritage crowdsourcing initiatives such as the Library of Congress's By the People [1], Smithsonian's Digital Volunteers [15], the United States Holocaust Memorial Museum's History Unfolded [9], Zooniverse [59], the New York Public Library's Emigrant City [4], the British Library's LibCrowds [10], the Living with Machines project [11], and Trove's newspaper crowdsourcing initiative [19], there are many opportunities to utilize crowdsourced data for machine learning tasks relevant to cultural heritage, from handwriting recognition to botany taxonomic classification [49,56]. These partnerships have the potential to provide insight into project design, decisions, workflows, and the context of the materials for which crowdsourcing contributions are sought.…”