Fine scale prediction of ecological community composition using a two-step sequential Machine Learning ensemble

Abstract

Prediction is one of the last frontiers in ecology. Indeed, predicting fine-scale species composition in natural systems is a complex challenge as multiple abiotic and biotic processes operate simultaneously to determine local species abundances. On the one hand, species intrinsic performance and their tolerance limits to different abiotic pressures modulate species abundances. On the other hand there is growing recognition that species interactions play an equally important role in limiting or promoting such abundances within ecological communities. Here, we present a joint effort between ecologists and data scientists to use data-driven models to predict species abundances using reasonably easy to obtain data. We propose a sequential data-driven modeling approach that in a first step predicts the potential species abundances based on abiotic variables, and in a second step uses these predictions to model the realized abundances once accounting for species competition. Using a curated data set over five years we predict fine-scale species abundances in a highly diverse annual plant community. Our models show a remarkable spatial predictive accuracy using only easy-to-measure variables in the field, yet such predictive power is lost when temporal dynamics are taken into account. This result suggests that predicting future abundances requires longer time series analysis to capture enough variability. In addition, we show that these data-driven models can also suggest how to improve mechanistic models by adding missing variables that affect species performance such as particular soil conditions (e.g. carbonate availability in our case). Robust models for predicting fine-scale species composition informed by the mechanistic understanding of the underlying abiotic and biotic processes can be a pivotal tool for conservation, especially given the human-induced rapid environmental changes we are experiencing. This objective can be achieved by promoting the knowledge gained with classic modelling approaches in ecology and recently developed data-driven models.

Predicting and prioritizing community assembly: learning outcomes via experiments

“…LOVE provides a promising approach for predicting coexistence outcomes from experimental community assembly datasets, and complements a growing body of work on machine learning approaches in community ecology (7,(17)(18)(19)(24)(25)(26). Applications to experiment prioritization seem possible and could be useful if explored ethically.…”

“…For LOVE, we did not perform further hyperparameter tuning (37) or explore other machine learning algorithms, e.g. support vector machines or boosted regression trees (26), as our focus was conceptual rather than algorithmic (cf. (38) for species distribution model algorithm comparisons).…”

“…These approaches still have relatively high observational data requirements (18,(23)(24)(25)(26). For instance, accurate predictions of spatial variation in abundance for 23 plant species across a 1 km transect, required 37,240 observations in (26). In contrast, a recent approach focusing on predicting abundance outcomes (7) has much lower data requirements (dozens to hundreds of observations) via a matrix pseudo-inversion approach, but cannot account for HOIs and is computationally infeasible in high richness communities.…”

“…Conceptually similar are machine learning methods such as random forest and neural networks that predict species abundance across space/time based on statistical relationships for observed species abundances. These approaches still have relatively high observational data requirements (18,(23)(24)(25)(26). For instance, accurate predictions of spatial variation in abundance for 23 plant species across a 1 km transect, required 37,240 observations in (26).…”

Structural asymmetry in biotic interactions as a tool to understand and predict ecological persistence

“…These mismatches between understanding ecological processes and predicting ecological dynamics are due to multiple causes, yet a common limitation is the lack of correspondence between the mathematical tools used for understanding and those used for predicting. For instance, in community ecology, while the mechanistic understanding of complex dynamics of interacting species uses well-established population models (Bimler et al ., 2018; Godoy & Levine, 2014), predictive studies are much less developed, and generally use a completely different set of tools including black-box approaches such as machine learning and neural networks (Civantos-Gómez et al ., 2021; Evans et al ., 2011).…”