The effects of small sample size and sample bias on threshold selection and accuracy assessment of species distribution models

Abstract

Species distribution models are used for a range of ecological and evolutionary questions, but often are constructed from few and/or biased species occurrence records. Recent work has shown that the presence‐only model Maxent performs well with small sample sizes. While the apparent accuracy of such models with small samples has been studied, less emphasis has been placed on the effect of small or biased species records on the secondary modeling steps, specifically accuracy assessment and threshold selection, particularly with profile (presence‐only) modeling techniques. When testing the effects of small sample sizes on distribution models, accuracy assessment has generally been conducted with complete species occurrence data, rather than similarly limited (e.g. few or biased) test data. Likewise, selection of a probability threshold – a selection of probability that classifies a model into discrete areas of presences and absences – has also generally been conducted with complete data. In this study we subsampled distribution data for an endangered rodent across multiple years to assess the effects of different sample sizes and types of bias on threshold selection, and examine the differences between apparent and actual accuracy of the models. Although some previously recommended threshold selection techniques showed little difference in threshold selection, the most commonly used methods performed poorly. Apparent model accuracy calculated from limited data was much higher than true model accuracy, but the true model accuracy was lower than it could have been with a more optimal threshold. That is, models with thresholds and accuracy calculated from biased and limited data had inflated reported accuracy, but were less accurate than they could have been if better data on species distribution were available and an optimal threshold were used.

Coupling of pollination services and coffee suitability under climate change

“…Maxent output provides a continuous probability map of species presence. We selected a threshold value to define suitable and unsuitable areas for bee species based on equal errors in sensitivity (proportion of accurately predicted presences) and specificity (proportion of absences accurately predicted; Table S1), as recommended by comparative studies (30).…”

Model Thresholds are More Important than Presence Location Type: Understanding the Distribution of Lowland tapir (Tapirus Terrestris) in a Continuous Atlantic Forest of Southeast Brazil

“…With appropriate sampling, model building, and threshold selection, it is possible to gain an understanding of local scale distributions from presence-only locations, yet what these distributions actually represent remains unclear [1][2]7,36].…”

“…Novel models can generate accurate and informative predictions from presence-only locations for a variety of faunal and floral species [1][2][5][6], but studies also highlight that model predictions are sensitive to a number of analytic and sampling biases [3,7].…”

Accuracy assessment model for classification result of remote sensing image based on spatial sampling

“…They are widely used in a great number of fields, including predicting epidemiology and burned area, 1,2 detecting forest and cultivated land changes, monitoring soil erosion and environmental change, 3,4 and mapping land cover and species distribution. [5][6][7][8] In particular, the majority of a remote sensing image should be conducted image classification before their applications, which can be achieved by either visual or computer-aided analysis. A key concern during image classification is whether the classification result derived from the remote sensing image has sufficient quality for operational application.…”