Automated synthesis of biodiversity knowledge requires better tools and standardised research output

Abstract

As the impact of anthropogenic activity on the environment has grown, research into biodiversity change and associated threats has also accelerated. Synthesising this vast literature is important for understanding the drivers of biodiversity change and identifying those actions that will mitigate further ecological losses. However, keeping pace with an ever‐increasing publication rate presents a substantial challenge to efficient syntheses, an issue which could be partly addressed by increasing levels of automation in the synthesis pipeline.Here, we evaluate the potential for automated tools to extract ecologically important information from the abstracts of articles compiled in the Living Planet Database. Specifically, we focused on extracting key information on taxonomy (studied species names), geographic location and estimated population trend, assessing the accuracy of automated versus manual information extraction, the potential for automated tools to introduce biases into syntheses, and evaluating if synthesising abstracts was enough to capture the key information from the full article.Taxonomic and geographic extraction tools performed reasonably well, although information on studied species was sometimes limited in the abstract (compared to the main text) preventing fast extraction. In contrast, extraction of trends was less successful, highlighting the challenges involved in automating information extraction from abstracts, such as deficiencies in the algorithms, linguistic complexity associated with ecological findings, and limited information when compared to the main text.In light of these results, we cautiously advocate for a wider use of automated taxonomic and geographic parsing tools for ecological synthesis. Additionally, to further the use of automated synthesis within ecology, we recommend a dual approach: development of improved computational tools to reduce biases; and enhanced protocols for abstracts (and associated metadata) to ensure key information is included in a format that facilitates machine‐readability.

Large language models help facilitate the automated synthesis of information on potential pest controllers

“…To put some of the results obtained in this work into a broader perspective, a comparison can be made with previous studies that have utilised text mining tools for applications in ecology. Previous attempts to extract taxonomic terms from abstracts have resulted in mean recall scores per abstract of 79.5% (Millard et al, 2020) and 93.6% (Cornford et al, 2022) with the help of the R package Taxize (Chamberlain and Szöcs, 2013), while the extraction of geographic locations has been achieved with a mean recall of 82.1% per abstract (Cornford et al, 2022) with the help of the CLIFFCLAVIN geoparser model (D’Ignazio et al, 2014). Furthermore, an extensive comparison of eight taxonomic named entity recognition (NER) models over four gold standard ecology corpora (Le Guillarme and Thuiller, 2022) reports scores for approximate matches ranging between 78–96% (precision), 74–93% (recall) and 76– 91% (F1-score).…”

“…Much of our knowledge of the global loss of biodiversity stems from large-scale syntheses of the ecological literature (Cornford et al, 2022). Such syntheses underlie the establishment of global environmental databases such as the WWF’s Living Planet Index (LPI, 2024), the PREDICTS (Hudson et al, 2017), and the BioTIME (Dornelas et al, 2018) databases, as well as the myriad global reports such as those of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES, 2019) and the Living Planet Report (Almond et al, 2022).…”

“…Text mining and natural language processing (NLP) methods are expected to have significant potential in automating tasks such as document classification, named entity recognition and disambiguation, and the extraction of relations between entities (Farrell et al, 2022). Previous approaches have focused on the identification of species in the text using probabilistic machine learning algorithms (Akella et al, 2012) or dictionarybased approaches (Gerner et al, 2010); the extraction of taxonomic terms and geographic locations (Millard et al, 2020;Cornford et al, 2022); the extraction of taxonomic terms using deep learning (Le Guillarme and Thuiller, 2022); the extraction of population trends using random forest and neural network classifiers (Cornford et al, 2022); and the classification of relevant and non-relevant scientific articles using logistic regression and convolutional neural network approaches (Cornford et al, 2021). Within the domain of deep learning (DL) for NLP tasks, large language models (LLMs) have been the subject of considerable attention in recent years by virtue of great leaps in progress and a wide range of potential applications (OpenAI, 2023a).…”

Large language models help facilitate the automated synthesis of information on potential pest controllers

“…To put some of the results obtained in this work into a broader perspective, we provide a comparison with previous studies that have utilised text mining tools for applications in ecology. Previous attempts to extract taxonomic terms from abstracts have resulted in mean recall scores per abstract of 79.5% (Millard et al., 2020) and 93.6% (Cornford et al., 2022) with the help of the R package Taxize (Chamberlain & Szöcs, 2013), while the extraction of geographic locations has been achieved with a mean recall of 82.1% per abstract (Cornford et al., 2022) with the help of the CLIFF‐CLAVIN geoparser model (D'Ignazio et al., 2014). Furthermore, an extensive comparison of eight taxonomic NER models over four gold standard ecology corpora (Le Guillarme & Thuiller, 2022) reports scores for approximate matches ranging between 78%–96% (precision), 74%–93% (recall) and 76%–91% (F1‐score).…”

“…Text mining and natural language processing (NLP) methods are expected to have significant potential in automating tasks such as document classification, named entity recognition (NER) and disambiguation, and the extraction of relations between entities (Farrell et al., 2022). Previous approaches in ecology have focused heavily on named entity recognition of species and taxonomy (Akella et al., 2012; Gerner et al., 2010; Le Guillarme & Thuiller, 2022; Millard et al., 2020) as well as geographical locations or population trends (Cornford et al., 2022) but also include document classification (Cornford et al., 2021) and relation extraction (Kaur et al., 2019). Furthermore, various gold standard databases of species names and taxonomy have been published to aid the evaluation of NER approaches in ecology (Abdelmageed et al., 2022; Nguyen et al., 2019).…”