A comparison of machine learning and statistical species distribution models: Quantifying overfitting supports model interpretation

Ramampiandra, Emma Chollet; Scheidegger, Andreas; Wydler, Jonas; Schuwirth, Nele

doi:10.1016/j.ecolmodel.2023.110353

Cited by 14 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generalized additive models (GAM) is a regression technique that applies smooth functions to assess the effect of predictors on response variables. Compared to other species distribution modelling (SDM) algorithms, machine learning algorithms are suitable for predicting complex relationships (Ramampiandra et al., 2023) with RF and BRT known to provide high prediction performances, while GAM is a highly flexible regression approach suitable for modelling data that do not require interactive terms (Elith et al., 2006). RF was applied using the ‘randomforest’ r package (Liaw & Wiener, 2002), BRT using the ‘gbm’ r package (Greenwell et al., 2019) and GAM using the ‘mgcv’ r package (Wood, 2017).…”

Section: Methodsmentioning

confidence: 99%

Moving with your mutualist: Predicted climate‐induced mismatch between Proteaceae species and their avian pollinators

Adedoja,

Dormann,

Coetzee

et al. 2024

Journal of Biogeography

View full text Add to dashboard Cite

AimClimate change influences species distribution in space and time, but predicting the overlap in the range of interacting species under different climate scenarios remains a challenge. Here, we explore how climate change influences shifts in species ranges among mutualists.LocationCape Floristic Region (CFR), South Africa.TaxonProteaceae and Passeriformes.MethodsWe used machine‐learning algorithms (random forest and boosted‐regression trees) and regression model (generalized additive models) to predict range shifts of 11 bird‐pollinated Proteaceae species, combined occurrence prediction of 71 bird‐pollinated Proteaceae and their two most important, endemic, pollinator bird species. We determined the degree of overlap in geographical ranges of nectar‐feeding birds and Proteaceae under different climate scenarios. Species ranges were projected to the years 2050 and 2070 using representative concentration pathways (RCP) 4.5 and 8.5 and three global climate models.ResultsThe majority of Proteaceae species in our model are predicted to experience range contractions, which ranged from 1% under 2050 RCP 4.5 CCSM4 to 79% under 2070 RCP 8.5 CNRM‐CM5 climate scenarios, leading to 55% and 62% range loss for Cape sugarbird and orange‐breasted sunbird, respectively, under extreme climate scenarios. Proteaceae species are predicted to experience least overlap with nectar‐feeding birds in the northern and eastern range under future climate scenarios.Main ConclusionClimate change threatens species occupying the mountain range of the northern limit and other regions of the CFR. Reduced range overlap of mutualists may have significant implications for the reproduction and persistence of Proteaceae. We suggest active monitoring of Proteaceae populations in regions where species are predicted to lose their range, particularly so for threatened species with small ranges.

show abstract

Section: Methodsmentioning

confidence: 99%

Moving with your mutualist: Predicted climate‐induced mismatch between Proteaceae species and their avian pollinators

Adedoja,

Dormann,

Coetzee

et al. 2024

Journal of Biogeography

View full text Add to dashboard Cite

show abstract

“…The application of TAVAC will allow for robust model interpretation in deep learning models leveraging commonly available H&E-stained images-especially those digitized through modern digital pathology techniquesfor high-resolution tumor status or subtype prediction. Overfitting affects model interpretability (Ramampiandra & Chollet, 2023). TAVAC is the first algorithm focused on the generalization of deep learning model interpretation.…”

Section: Application To Two Independent Breast Cancer Pathological Im...mentioning

confidence: 99%

“…At the same time, the generalization of deep learning model interpretation through attention visualization has been understudied. Ramampiandra & Chollet previously commented that overfitted models impede the interpretation of machine learning models (Ramampiandra & Chollet, 2023), yet no solution is proposed to solve the problem. Model interpretation for ViT models is carried out through analysis of attention layers, which are the weight on the input representing the focus of the model (Dosovitskiy, et al, 2020;Abnar & Zuidema, 2020).…”

mentioning

confidence: 99%

Quantifying Interpretation Reproducibility in Vision Transformer Models with TAVAC

Zhao,

Agyemang,

Liu

et al. 2024

Preprint

View full text Add to dashboard Cite

The use of deep learning algorithms to extract meaningful diagnostic features from biomedical images holds the promise to improve patient care given the expansion of digital pathology. Among these deep learning models, Vision Transformer (ViT) models have been demonstrated to capture long-range spatial relationships with more robust prediction power for image classification tasks than regular convolutional neural network (CNN) models, and also better model interpretability. Model interpretation is important for understanding and elucidating how a deep learning model makes predictions, especially for developing transparent models for digital pathology. However, like other deep learning algorithms, with limited annotated biomedical imaging datasets, ViT models are prone to poor performance due to overfitting, which can lead to false predictions due to random noise. Overfitting affects model interpretation when predictions are made out of random noise. To address this issue, we introduce a novel metric, Training Attention and Validation Attention Consistency (TAVAC), for evaluating ViT model degree of overfitting on imaging datasets and quantifying the reproducibility of interpretation. Specifically, the model interpretation is performed by comparing the high-attention regions in the image between training and testing. We test the method on four publicly available image classification datasets and two independent breast cancer histological image datasets. All overfitted models exhibited significantly lower TAVAC scores than the good-fit models. The TAVAC score quantitatively measures the level of generalization of model interpretation on a fine-grained level for small groups of cells in each H&E image, which cannot be provided by traditional performance evaluation metrics like prediction accuracy. Furthermore, the application of TAVAC extends beyond medical diagnostic AI models; it enhances the monitoring of model interpretative reproducibility at pixel-resolution in basic research, to reveal critical spatial patterns and cellular structures essential to understanding biological processes and disease mechanisms. TAVAC sets a new standard for evaluating the performance of deep learning model interpretation and provides a method for determining the significance of high-attention regions detected from the attention map of the biomedical images.

show abstract

“…When a statistical model is overly intricate and catches noise in the data rather than underlying patterns, overfitting occurs. As a result, new data may not generalize well, and model interpretability may suffer [9]. Predictive models are created using machine learning methods, including decision trees [10], random forests [11], neural networks [12], and support vector machines [13].…”

Section: Introductionmentioning

confidence: 99%

Presenting a Hybrid Method to Overcome the Challenges of Determining the Uncertainty of Future Stock Price Identification

Zou,

Xiao

2024

IJACSA

View full text Add to dashboard Cite

A particular location, framework, or forum where buyers and sellers congregate to trade products, services, or assets is referred to as an economic market. While the future is unpredictable and unknowable, it is still possible to make informed predictions about the course of events. Predicting stock market movements using artificial intelligence and machine learning is one such potential. Even if the stock market is volatile, it is still feasible and wise to use artificial intelligence to create well-informed forecasts before making an investment. The current work suggests a novel approach to increase stock price forecast accuracy by integrating the Radical basis function with Particle Swarm Optimization, Slime Mold Algorithm, and Moth Flame Optimization. The objective of the study is to improve stock price forecast accuracy while accounting for the complexity and volatility of financial markets. The efficacy of the proposed strategy has been tested in the real world using historical stock price statistics. Results demonstrate considerable accuracy improvements over traditional RBF models. The combined strength of RBF and the optimization technique enhances the model's ability to adapt to changing market conditions in addition to increasing prediction accuracy. Results were 0.984, 0.990, 0.991, and 0.994 for RBF, PSO-RBF, SMA, and MFO-RBF, respectively. The performance of MFO-RBF in comparison to RBF shows how combining with the optimizer can enhance the performance of the given model. By contrasting the outcomes of various optimizers, the most accurate optimization has been determined as the main optimizer of the model.

show abstract

A comparison of machine learning and statistical species distribution models: Quantifying overfitting supports model interpretation

Cited by 14 publications

References 42 publications

Moving with your mutualist: Predicted climate‐induced mismatch between Proteaceae species and their avian pollinators

Moving with your mutualist: Predicted climate‐induced mismatch between Proteaceae species and their avian pollinators

Quantifying Interpretation Reproducibility in Vision Transformer Models with TAVAC

Presenting a Hybrid Method to Overcome the Challenges of Determining the Uncertainty of Future Stock Price Identification

Contact Info

Product

Resources

About