Testing whether ensemble modelling is advantageous for maximising predictive performance of species distribution models

Abstract

Predictive performance is important to many applications of species distribution models (SDMs). The SDM ‘ensemble’ approach, which combines predictions across different modelling methods, is believed to improve predictive performance, and is used in many recent SDM studies. Here, we aim to compare the predictive performance of ensemble species distribution models to that of individual models, using a large presence–absence dataset of eucalypt tree species. To test model performance, we divided our dataset into calibration and evaluation folds using two spatial blocking strategies (checkerboard‐pattern and latitudinal slicing). We calibrated and cross‐validated all models within the calibration folds, using both repeated random division of data (a common approach) and spatial blocking. Ensembles were built using the software package ‘biomod2’, with standard (‘untuned’) settings. Boosted regression tree (BRT) models were also fitted to the same data, tuned according to published procedures. We then used evaluation folds to compare ensembles against both their component untuned individual models, and against the BRTs. We used area under the receiver‐operating characteristic curve (AUC) and log‐likelihood for assessing model performance. In all our tests, ensemble models performed well, but not consistently better than their component untuned individual models or tuned BRTs across all tests. Moreover, choosing untuned individual models with best cross‐validation performance also yielded good external performance, with blocked cross‐validation proving better suited for this choice, in this study, than repeated random cross‐validation. The latitudinal slice test was only possible for four species; this showed some individual models, and particularly the tuned one, performing better than ensembles. This study shows no particular benefit to using ensembles over individual tuned models. It also suggests that further robust testing of performance is required for situations where models are used to predict to distant places or environments.

Species distribution modelling of Bryde’s whales, humpback whales, southern right whales, and sperm whales in the southern African region to inform their conservation in expanding economies

“…Our study, and a study done by Hao et al (2020), indicates that ensemble models do not always outperform single algorithms as they produce varied results. Hao et al (2020) suggest that ensemble models can be outperformed by single algorithms when predicting to distant areas and when single algorithms are finely tuned. In this study, the RF algorithms, on average, produced the highest AUC scores (Table S2).…”

“…For example, by spatially thinning the presence data, some but not all spatial biases were removed. Also, to prevent overfitting and improve this study, spatial blocks should be used, similar to the study by Hao et al (2020). Overall, the use of ensemble models does not necessarily provide the best model performance as individual algorithms can produce AUC values that are as high, or higher than ensemble models.…”

“…In this study, the RF algorithms, on average, produced the highest AUC scores (Table S2). RF algorithms in this study have complex trees with no spatial blocking during validation and are, therefore, prone to overfitting (Hao et al, 2020). Looking at Fig.…”

Multi-scale habitat selection and impacts of climate change on the distribution of four sympatric meso-carnivores using random forest algorithm

“…The "ensemble" modeling approach, which combines predictions across different modeling approaches, is generally regarded to perform better than single modeling approaches. However, a recent study compared the predictive performance of ensemble distribution models to that of individual models and they report no particular benefit to using ensemble modeling approaches over individually tuned models (Hao et al 2020). The high predictive success and inclusion of scale-optimized variables that drive the distribution of meso-carnivores suggest that these multi-scale niche models may tightly reflect the realized niches of these four species.…”

Klasifikasi Untuk Prediksi Cuaca Menggunakan Esemble Learning

“…Menggunakan daerah di bawah receiver-Operating Curve karakteristik (AUC) dan log-likelihood untuk menilai kinerja model. Hasil ensemble model dengan baik, tetapi tidak konsisten lebih baik daripada komponen untuned model individu atau BRTs [3].…”