A Tale of Two Crowds: Public Engagement in Plankton Classification

Abstract

"Big data" are becoming common in biological oceanography with the advent of sampling technologies that can generate multiple, high-frequency data streams. Given the need for "big" data in ocean health assessments and ecosystem management, identifying and implementing robust, and efficient processing approaches is a challenge for marine scientists. Using a large plankton imagery data set, we present two crowd-sourcing approaches applied to the problem of classifying millions of organisms. The first used traditional crowd-sourcing by asking the public to identify plankton through a web-interface. The second challenged the data science community to develop algorithms via an industry partnership. We found traditional crowd-sourcing was an excellent way to engage and educate the public while crowd-sourcing data scientists rapidly generated multiple, effective solutions. As the need to process and visualize large and complex marine data sets is expected to grow over time, effective collaborations between oceanographers and computer and data scientists will become increasingly important.

Improving plankton image classification using context metadata

“…Recently, CNNs have been applied to plankton classification problems hinting at the potential of the approach (Wang et al ; Zheng et al ). A public competition (Robinson et al ) stimulated new solutions (Dieleman et al ) but there has not yet been a quantitative assessment of specific design choices when considering a CNN for plankton image analysis. Luo et al () validated the findings from the contest and showed that CNNs do successfully generalize to future images and therefore can be used as part of an end‐to‐end workflow that they outline in detail.…”

“…Ensembling of feature‐based models without metadata on plankton images can be beneficial (Ellen et al ). The concept of ensembling is well accepted, as nearly every major machine learning competition is won by an ensemble of multiple models (Robinson et al ). The dynamics of an ensemble make academic analysis difficult, because the efficacy of each model needs to be examined as well as the effects of interactions between them, but evidence supports their implementation.…”

Prey and predator overlap at the edge of a mesoscale eddy: fine-scale, in-situ distributions to inform our understanding of oceanographic processes

“…With a focus on these rare plankton, the volume of water that ISIIS images is orders of magnitude higher than that of other imaging systems (e.g., VPR 32 , LOKI 34,37 ). ISIIS's large imaging frame, with a 13 ×13-cm field of view and 50 cm depth of field allows for the undisturbed imaging of a variety of plankton types including fragile gelatinous zooplankton 29,41,42,76 . The resulting images have a pixel resolution of 66 μm.…”

Machine learning from crowds: A systematic review of its applications

“…There are a great number of applications in other domains, such as security or energy, which are also relevant for this study (Table ): deduplication of digital libraries (Georgescu et al, ); imitation learning (Chung et al, ); evaluation of procedural content generation (Roberts & Chen, ); action model acquisition (Zhuo, ); weighting antivirus labels (Kantchelian et al, ); aerosol optical depth estimation (Djuric et al, ); point of interest labeling (Hu et al, ); detection of spatial events (Ouyang et al, ); interstate conflict measurement (D'Orazio et al, ); annotation of energy data (Cao et al, ); extracting semantic attributes to describe concepts (Tian et al, ); category learning (Danileiko & Lee, ); crowd databases (Robinson et al, ); and network quality measurements (Li, Gao, et al, ).…”