Endless Forams: >34,000 Modern Planktonic Foraminiferal Images for Taxonomic Training and Automated Species Recognition Using Convolutional Neural Networks

Abstract

Planktonic foraminiferal species identification is central to many paleoceanographic studies, from selecting species for geochemical research to elucidating the biotic dynamics of microfossil communities relevant to physical oceanographic processes and interconnected phenomena such as climate change. However, few resources exist to train students in the difficult task of discerning amongst closely related species, resulting in diverging taxonomic schools that differ in species concepts and boundaries. This problem is exacerbated by the limited number of taxonomic experts. Here we document our initial progress toward removing these confounding and/or rate-limiting factors by generating the first extensive image library of modern planktonic foraminifera, providing digital taxonomic training tools and resources, and automating species-level taxonomic identification of planktonic foraminifera via machine learning using convolution neural networks. Experts identified 34,640 images of modern (extant) planktonic foraminifera to the species level. These images are served as species exemplars through the online portal Endless Forams (endlessforams.org) and a taxonomic training portal hosted on the citizen science platform Zooniverse (zooniverse.org/projects/ahsiang/ endless-forams/). A supervised machine learning classifier was then trained with~27,000 images of these identified planktonic foraminifera. The best-performing model provided the correct species name for an image in the validation set 87.4% of the time and included the correct name in its top three guesses 97.7% of the time. Together, these resources provide a rigorous set of training tools in modern planktonic foraminiferal taxonomy and a means of rapidly generating assemblage data via machine learning in future studies for applications such as paleotemperature reconstruction.

Automated analysis of foraminifera fossil records by image classification using a convolutional neural network

“…morphology) (Krizhevsky et al, 2012) and is on par with human performance (Russakovsky et al, 2015). Current popular networks include VGG (Simonyan and Zisserman, 2015), Inception (Szegedy et al, 2015(Szegedy et al, , 2016, ResNet (He et al, 2016a, b;Zagoruyko and Komodakis, 2016;Xie et al, 2017) and DenseNet (Huang et al, 2017).…”

“…In the foraminifera domain, one current approach is using transfer learning with pre-trained ResNet and VGG networks to classify foraminifera images coloured according to 3D cues from 16-way lighting (Zhong et al, 2018;Mitra et al, 2019). Hsiang et al (2019) constructed a large planktonic foraminifera image set, Endless Forams, consisting of over 27 000 images classified into 35 species classed by multiple expert taxonomists. They then applied transfer learning using the VGG network to compare CNN-based classification of this dataset with human performance.…”

Quantifying the Effect of Anthropogenic Climate Change on Calcifying Plankton

“…We analysed 10 ml aliquots from the 180-2000 um size fraction, samples were stored in a freezer in ethanol. Detailed accounts of the collection methodology can be found in Pesant et al 2015. Species identifications follow the existing understanding of modern foraminiferal taxonomy [37][38][39] . The primary classification is based on the chamber arrangement, wall structure and principally spinose or non-spinose ornamentation 40,41 .…”

Time Machine Biology: Cross-Timescale Integration of Ecology, Evolution, and Oceanography

“…Automated picking systems that take sieved size fractions, separate them into individual particles, and image each particle may greatly reduce picking times (de Garidel-Thoron et al, 2017;Itaki et al, 2020). With samples that have already been picked and mounted, hundreds or thousands of individual microfossils can be imaged simultaneously in three dimensions and algorithmically parsed into individual images from which basic morphometric traits and features can be automatically extracted and ana-lyzed at the assemblage scale (Beaufort et al, 2014;Elder et al, 2018;Hsiang et al, 2018Hsiang et al, , 2019Kahanamoku et al, 2018).…”

“…These efforts build on decades of previous automation work that either extracted coarser (size related) data or was relatively more labor intensive (Bollmann et al, 2005;Knappertsbusch et al, 2009). Given sufficient training data sets, convolutional neural nets can now identify planktonic foraminifera, coccolithophores, and radiolarians with accuracy similar to that of taxonomic specialists (Beaufort and Dollfus, 2004;de Garidel-Thoron et al, 2017;Hsiang et al, 2019;Itaki et al, 2020). Given these ongoing developments, it is becoming possible to envision a near future in which the entire sample processing and data extraction workflow is streamlined and largely automated, with taxonomic experts guiding and overseeing the process but spending the majority of their time analyzing data sets that may be far larger, denser, and more data-rich than is currently feasible.…”