The effect of automated taxa identification errors on biological indices

rje, Johanna; Krkkinen, Salme; Meissner, Kristian; Iosifidis, Alexandros; İnce, Türker; Gabbouj, Moncef; Kıranyaz, Serkan

doi:10.1016/j.eswa.2016.12.015

Cited by 6 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most of the previous studies have focused on single-image data (see e.g. Ärje et al, 2010;Kiranyaz et al, 2011;Ärje et al, 2013;Joutsijoki et al, 2014;Uusitalo et al, 2016;Lee et al, 2016;Ärje et al, 2017) and have not taken the inherent hierarchical structure of the data into account. In single-image data studies, the posture of the specimens can have substantial impact on the classification.…”

Section: Ecological Assessmentmentioning

confidence: 99%

Human experts vs. machines in taxa recognition

Ärje,

Raitoharju,

Iosifidis

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

The step of expert taxa recognition currently slows down the response time of many bioassessments. Shifting to quicker and cheaper state-of-the-art machine learning approaches is still met with expert scepticism towards the ability and logic of machines. In our study, we investigate both the differences in accuracy and in the identification logic of taxonomic experts and machines. We propose a systematic approach utilizing deep Convolutional Neural Nets with the transfer learning paradigm and extensively evaluate it over a multi-pose taxonomic dataset with hierarchical labels specifically created for this comparison. We also study the prediction accuracy on different ranks of taxonomic hierarchy in detail. Our results revealed that human experts using actual specimens yield the lowest classification error (CE = 6.1%). However, a much faster, automated approach using deep Convolutional Neural Nets comes close to human accuracy (CE = 11.4%). Contrary to previous findings in the literature, we find that for machines following a typical flat classification approach commonly used in machine learning performs better than forcing machines to adopt a hierarchical, local per parent node approach used by human taxonomic experts. Finally, we publicly share our unique dataset to serve as a public benchmark dataset in this field.

show abstract

Section: Ecological Assessmentmentioning

confidence: 99%

Human experts vs. machines in taxa recognition

Ärje,

Raitoharju,

Iosifidis

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…They found a great discrepancy between the experts determining the true taxonomic classes and the audited laboratory workers. Contrastingly, in a study on the effect of mistakes made in automated taxa identification on biological indices, Ärje et al [2] found a relatively small impact. Literature on direct human versus machine comparisons in classification tasks in an aquatic biomonitoring context is equally scant and ambiguous.…”

Section: Sample Identificationmentioning

confidence: 89%

“…However, most of the previous studies have focused on single-image data [see e.g. 3,20,4,17,35,22,2] and have not taken the inherent hierarchical structure of the data into account. In single-image data studies, the posture of the specimens can have substantial impact on the classification.…”

Section: Sample Identificationmentioning

confidence: 99%

Human experts vs. machines in taxa recognition

Ärje

Raitoharju

Iosifidis

et al. 2020

Signal Processing: Image Communication

Self Cite

View full text Add to dashboard Cite

“…Moreover, these differences are not always evident even for a human expert [9,62,63]. In this sense, the accuracy of automatic classification results is critical to avoid affecting biological evaluation indices and to reduce the bias caused by automation in classification methods [59].…”

Section: Phase 1: Quantitative Analysis Of Retrieved Data and Assigne...mentioning

confidence: 99%

“…In some instances, SVM algorithms have been optimized based on class selection and splitting methods such as one-vs.-one, one-vs.-all [49], or half-against-half [54][55][56]. Likewise, Bayesian algorithms, including random forest (RF), random Bayes forest (RBF) [52], or random Bayes array (RBA), have allowed the application of class division, in some cases, regarding taxonomic information [57][58][59].…”

mentioning

confidence: 99%

Computer-Assisted Bioidentification Using Freshwater Macroinvertebrates: A Scoping Review

Cruz-Cruz

López

Vargas-Cañas

et al. 2022

Water

View full text Add to dashboard Cite

Background: Evaluation and prediction of the freshwater status based on freshwater macroinvertebrates (FwM) has become valuable in bioindication because they provide a more general and accurate picture of the ecological status of water bodies over time. Recent research on bioindication through FwM has increased the use of computational technologies, mainly in the classification and data analysis stages of water quality assessment and prediction. Objective: This scoping review aims to provide an overview of different approaches in computer-assisted bioindication with FwM. Particularly, the objective is to identify the techniques and strategies employed for FwM automatic classification or data treatment, characterize their use in recent years, and discuss gaps and challenges to broaden the scope of bioindication as a tool for understanding real conditions in a water body. Design: The scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) extension for scoping reviews (ScR). Scopus and Web of Science databases were used to identify articles published between 1999 and 2022. We selected 81 publications that used computational technology for automatic FwM classification or data analysis to predict water quality using biological indices. Results and conclusions: We identified two areas of applying computational technologies in bioindication studies with FwM. Firstly, computer-assisted technologies are used to evaluate water quality through samples already classified by human experts which correspond to 57% of the documents analyzed. The second application area is the automatic classification of FwM. In addition, we determined the main critical factors affecting strategy selection in each of the studies, such as taxonomic resolution, sample size and quality, image quality, data size, and complexity. Finally, we established the relationship between the strategies and algorithms employed in a timeline for automatic classification according to available FwM image databases. The research will allow taxonomic and related experts to better understand the role of computational technologies in FwM studies and thus increase confidence in these techniques to extend their use in bioassessment tasks.

show abstract

The effect of automated taxa identification errors on biological indices

Abstract: This is an electronic reprint of the original article. This reprint may differ from the original in pagination and typographic detail.

Cited by 6 publications

References 39 publications

Human experts vs. machines in taxa recognition

Human experts vs. machines in taxa recognition

Human experts vs. machines in taxa recognition

Computer-Assisted Bioidentification Using Freshwater Macroinvertebrates: A Scoping Review

Contact Info

Product

Resources

About