Assessment of a large number of empirical plant species niche models by elicitation of knowledge from two national experts

Abstract

Quantitative models play an increasing role in exploring the impact of global change on biodiversity. To win credibility and trust, they need validating. We show how expert knowledge can be used to assess a large number of empirical species niche models constructed for the British vascular plant and bryophyte flora. Key outcomes were (a) scored assessments of each modeled species and niche axis combination, (b) guidance on models needing further development, (c) exploration of the trade‐off between presenting more complex model summaries, which could lead to more thorough validation, versus the longer time these take to evaluate, (d) quantification of the internal consistency of expert opinion based on comparison of assessment scores made on a random subset of models evaluated by both experts. Overall, the experts assessed 39% of species and niche axis combinations to be “poor” and 61% to show a degree of reliability split between “moderate” (30%), “good” (25%), and “excellent” (6%). The two experts agreed in only 43% of cases, reaching greater consensus about poorer models and disagreeing most about models rated as better by either expert. This low agreement rate suggests that a greater number of experts is required to produce reliable assessments and to more fully understand the reasons underlying lack of consensus. While area under curve (AUC) statistics showed generally very good ability of the models to predict random hold‐out samples of the data, there was no correspondence between these and the scores given by the experts and no apparent correlation between AUC and species prevalence. Crowd‐sourcing further assessments by allowing web‐based access to model fits is an obvious next step. To this end, we developed an online application for inspecting and evaluating the fit of each niche surface to its training data.

Model‐based assessment of the impact of agri‐environment scheme options and short‐term climate change on plant biodiversity in temperate grasslands

“…Note that this is a strong test since the baseline data are wholly independent of the model training data. Also, see Smart et al (2019) for further testing of MultiMOVE. We consider that these results indicate a useful level of model performance in predictive mode.…”

“…We used the MultiMOVE R package (Henrys et al, 2015) as the source of ENMs for higher and lower plants in the British flora. The package has been tested and applied in a number of studies under a range of contrasting scenarios (De Vries et al, 2010; Emmett & the GMEP team, 2017; Henrys et al, 2015; Rowe et al, 2015; Smart et al, 2019). In summary, it uses a small ensemble of five statistical methods to model the realised niche of 1262 taxa covering the most common and many less common plants and bryophytes (Henrys et al, 2015; Smart et al, 2019 for full description) in the British flora.…”

“…We used the MultiMOVE R package (Henrys et al, 2015) as the source of ENMs for higher and lower plants in the British flora. The package has been tested and applied in a number of studies under a range of contrasting scenarios (De Vries et al, 2010;Emmett & the GMEP team, 2017;Henrys et al, 2015;Rowe et al, 2015;Smart et al, 2019). In summary, it uses a small ensemble of five statistical F I G U R E 1 Map of the Glastir Monitoring and Evaluation Programme (GMEP) 1 km squares surveyed between 2013 and 2016, survey squares are not shown to scale to preserve the confidentiality of their locations.…”

Causal inference and large‐scale expert validation shed light on the drivers of SDM accuracy and variance

“…Whilst most studies evaluate SDM accuracy using skill statistics, an alternative is to solicit expert opinion. For example, Smart et al (2019) sought expert opinion on the realism of species response curves estimated by small‐scale niche models for vascular plants and bryophytes in the United Kingdom (UK). Similarly, Beck et al (2014) sought expert opinion on the spatial predictions produced by various SDMs for a European butterfly.…”

Causal inference and large-scale expert validation shed light on the drivers of SDM accuracy and variance

“…Whilst most studies evaluate SDM accuracy using skill statistics, an alternative is to solicit expert opinion. For example, Smart et al (2019) sought expert opinion on the realism of species response curves estimated by small-scale niche models for vascular plants and bryophytes in the United Kingdom (UK). Similarly, Beck et al (2014) sought expert opinion on the spatial predictions produced by various SDMs for a European butterfly.…”