Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish Species

Leipzig, Jeremy; Bakis, Yasin; Wang, Xiaojun; Elhamod, Mohannad; Diamond, Kelly; Dahdul, Wasila; Karpatne, Anuj; Maga, A. Murat; Mabee, Paula M.; Bart, Henry L.; Greenberg, Jane

doi:10.1101/2021.01.28.428644

Cited by 5 publications

(14 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This point is also emphasized by Wieczorek et al [11] in their report on the variety of DwC metadata extensions needed to meet growing community concerns and requirements, including data quality and fitness. This point is addressed in detail by Leipzig et al [5], drawing from Tulane University's manual curation of 22 metadata properties that characterize digitized specimen image quality, and further motivates the research reported in this paper.…”

Section: R E L a T E Dw O R K 21 Metadata Standards And Approaches Fo...mentioning

confidence: 71%

“…A key impetus has been engagement of both teams in the NSF Biology Guided Neural Networks (BGNN) project, which is developing a novel class of artificial neural networks that aims to exploit machine readable, predictive knowledge associated with specimen images, phylogenies, and anatomical ontologies. Initial research successfully demonstrated computational approaches for creating image quality metadata [5]; and, further, that by combining ML and image informatics, researchers automatically determine image quality and metadata, such as fish quantity, location and orientation, and image scaling based on ruler identification and measurement [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computational metadata generation methods for biological specimen image collections

et al. 2022

View full text Add to dashboard Cite

Metadata is a key data source for researchers seeking to apply machine learning (ML) to the vast collections of digitized biological specimens that can be found online. Unfortunately, the available metadata is often sparse and, at times, erroneous. This paper extends previous research with the Illinois Natural History Survey (INHS) collection (7,244 specimen images) using computational approaches to analyze image quality, and then automatically generate 22 metadata properties representing the image quality and morphological features of the specimens. In the research reported here, we demonstrate the extension of our initial work to University of Wisconsin Zoological Museum (UWZM) collection (4,155 specimen images). Further, we enhance our computational methods in four ways: 1) augmenting the training set, 2) applying contrast enhancement, 3) upscaling small objects, and 4) refining of our processing logic. Together these new methods improved our overall error rates from 4.6% to 1.1%. These enhancements also allowed Computational Metadata Generation Methods us to compute an additional set of 17 image-based metadata properties. The new metadata properties provide supplemental features and information that may also be used to analyze and classify the fish specimens. Examples of these new features include convex area, eccentricity, perimeter, skew, etc. The newly refined process further outperforms humans in terms of time and labor cost, as well as accuracy, providing a novel solution for leveraging digitized specimens with ML. This research demonstrates the ability of computational methods to enhance the digital library services associated with the tens of thousands of digitized specimens stored in open-access repositories world-wide by generating accurate and valuable metadata for those repositories.

show abstract

Section: R E L a T E Dw O R K 21 Metadata Standards And Approaches Fo...mentioning

confidence: 71%

Section: Introductionmentioning

confidence: 99%

Computational metadata generation methods for biological specimen image collections

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Technicians employed by Tulane University have manually generated the 22 metadata properties deemed crucial to the overall BGNN project [9] for a large number of INHS images. 20, 699 total entries were created by 13 technicians that spanned 8, 398 unique images, of which 7, 244 were both not part of the detectron training set and met our current admissibility criteria for detectron and pixel processing.…”

Section: Resultsmentioning

confidence: 99%

“…Using ML and image informatics algorithms, it is able to locate, mask and analyze specimens (currently limited to fish) in collection images with a high degree of accuracy. It produces 6 of the 22 core BGNN metadata properties [9], as well as image contrast, bounding boxes, scale and length information. Testing this approach on 7, 244 images from the INHS dataset [4], we see that the vast majority of the resulting metadata is correct within a tolerance of a few percentage points, and in some cases contains fewer mistakes than the manually generated validation data.…”

Section: Discussionmentioning

confidence: 99%

“…Even so, metadata describing image quality is severely limited and generally missing. This point is addressed in detail by Leipzig et al [9] and serves as the rationale for Tulane University's effort to manually capture content for 22 metadata properties that characterize digitized specimen image quality. Their work is being conducted in connection with the larger BGNN initiative, and the difficulties encountered during the process underscore the need to explore automatic metadata generation methods.…”

Section: A Metadata For Natural History Image Collectionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Metadata Generation for Fish Specimen Image Collections

Pepper

Greenberg

Bakis

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Metadata are key descriptors of research data, particularly for researchers seeking to apply machine learning (ML) to the vast collections of digitized specimens. Unfortunately, the available metadata is often sparse and, at times, erroneous. Additionally, it is prohibitively expensive to address these limitations through traditional, manual means. This paper reports on research that applies machine-driven approaches to analyzing digitized fish images and extracting various important features from them. The digitized fish specimens are being analyzed as part of the Biology Guided Neural Networks (BGNN) initiative, which is developing a novel class of artificial neural networks using phylogenies and anatomy ontologies. Automatically generated metadata is crucial for identifying the high-quality images needed for the neural network's predictive analytics. Methods that combine ML and image informatics techniques allow us to rapidly enrich the existing metadata associated with the 7,244 images from the Illinois Natural History Survey (INHS) used in our study. Results show we can accurately generate many key metadata properties relevant to the BGNN project, as well as general image quality metrics (e.g. brightness and contrast). Results also show that we can accurately generate bounding boxes and segmentation masks for fish, which are needed for subsequent machine learning analyses. The automatic process outperforms humans in terms of time and accuracy, and provides a novel solution for leveraging digitized specimens in ML. This research demonstrates the ability of computational methods to enhance the digital library services associated with the tens of thousands of digitized specimens stored in open-access repositories worldwide.

show abstract