A ranking system for comparing models' performance combining multiple statistical criteria and scenarios: The case of reference evapotranspiration models

Aschonitis, Vassilis; Lekakis, Emmanuel; Tziachris, Panagiotis; Doulgeris, Charalampos; Papadopoulos, Frantzis; Papadopoulos, Aristotelis; Papamichail, Dimitris

doi:10.1016/j.envsoft.2019.01.005

Cited by 15 publications

(6 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this kind of research it is not advisable to use a single goodness-of-fit criterion, as highlighted in this study where the abcd model provides the best performance according to the NSE. But the final decision must be made based on the greatest number of possible criteria, and applying an objective method that allows ranking the models taking into account all these criteria [ 53 ]. In these methods, the criteria used must be standardized, in order to make them comparable, and the inner balance error committed by the model (ε) could be included among them.…”

Section: Discussionmentioning

confidence: 99%

Modulation of the goodness of fit in hydrological modelling based on inner balance errors

Pellicer-Martínez

Castillo

Portela³

et al. 2021

PLoS ONE

View full text Add to dashboard Cite

In hydrological modelling, a good result for the criterion of goodness of fit does not always imply that the hypothesis of mass conservation is fulfilled, and models can lose their essential physical soundness. We propose a way for detecting this anomaly by accounting the resulting water balance during model simulation and use it to modulate the obtained goodness of fit. We call this anomaly in water balance as “inner balance error of the model”. To modulate the goodness of fit values, a penalty function that depends on this error is proposed. In addition, this penalty function is introduced into a multi-criteria objective function, which is also tested. This procedure was followed in modelling the Headwater of the Tagus River (Spain), applying the monthly abcd water balance model. Modulation of the goodness of fit allowed for detecting balance errors in the modelling, revealing that in the simulation of some catchments the model tends to accumulate water in, or release water from, the reservoir that simulates groundwater storage. Although the proposed multi-criteria objective function solves the inner balance error for most catchments, in some cases the error cannot be corrected, indicating that any error in the input and output data is probably related to groundwater flows.

show abstract

Section: Discussionmentioning

confidence: 99%

Modulation of the goodness of fit in hydrological modelling based on inner balance errors

Pellicer-Martínez

Castillo

Portela³

et al. 2021

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…n-RMSE (%) values within the range of 0–10% are considered “excellent”, those between 10–20% are classified as “good”, values from 20–30% fall into the “fair” category, and any values exceeding 30% are deemed “poor” 45 , 46 . To rank the models by combining the results of calibration and validation of multiple statistical criteria, standardized Ranking Performance Index (sRPI) was calculated 47 . The sRPI ranges between 0 and 1, where the worst-performing model gets 0, while the best model gets value of 1.…”

Section: Methodsmentioning

confidence: 99%

Combining biophysical parameters with thermal and RGB indices using machine learning models for predicting yield in yellow rust affected wheat crop

Singh,

Krishnan,

Singh

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Evaluating crop health and forecasting yields in the early stages are crucial for effective crop and market management during periods of biotic stress for both farmers and policymakers. Field experiments were conducted during 2017–18 and 2018–19 with objective to evaluate the effect of yellow rust on various biophysical parameters of 24 wheat cultivars, with varying levels of resistance to yellow rust and to develop machine learning (ML) models with improved accuracy for predicting yield by integrating thermal and RGB indices with crucial plant biophysical parameters. Results revealed that as the level of rust increased, so did the canopy temperature and there was a significant decrease in crop photosynthesis, transpiration, stomatal conductance, leaf area index, membrane stability index, relative leaf water content, and normalized difference vegetation index due to rust, and the reductions were directly correlated with levels of rust severity. The yield reduction in moderate resistant, low resistant and susceptible cultivars as compared to resistant cultivars, varied from 15.9–16.9%, 28.6–34.4% and 59–61.1%, respectively. The ML models were able to provide relatively accurate early yield estimates, with the accuracy increasing as the harvest approached. The yield prediction performance of the different ML models varied with the stage of the crop growth. Based on the validation output of different ML models, Cubist, PLS, and SpikeSlab models were found to be effective in predicting the wheat yield at an early stage (55-60 days after sowing) of crop growth. The KNN, Cubist, SLR, RF, SpikeSlab, XGB, GPR and PLS models were proved to be more useful in predicting the crop yield at the middle stage (70 days after sowing) of the crop, while RF, SpikeSlab, KNN, Cubist, ELNET, GPR, SLR, XGB and MARS models were found good to predict the crop yield at late stage (80 days after sowing). The study quantified the impact of different levels of rust severity on crop biophysical parameters and demonstrated the usefulness of remote sensing and biophysical parameters data integration using machine-learning models for early yield prediction under biotically stressed conditions.

show abstract

“…where a, b, c, d, and e are regression coefficients. Similarly, the general form of Equation ( 8), with an intercept according to Ghare et al [61] for estimating ET o using only E pan , was defined as:…”

Section: Pan Evaporation Methods and General Forms For Local Conditionsmentioning

confidence: 99%

“…This approach also creates other problems that are associated to the regression analysis. During a regression analysis, an optimization algorithm aims at minimizing the errors between observed and predicted values of the dependent variable, and because of this procedure, the observed k p with larger values have a greater effect on the error measures [61] and, consequently, a greater effect on regulating the regression coefficients. For example, if the measured k p values of colder days (lower ET o ) are larger from the respective values of warmer days (higher ET o ), which is not unusual [29,36,49], then the final calibrated k p model will perform better at colder days with lower ET o values.…”

Section: Introductionmentioning

confidence: 99%

Assessing the Robustness of Pan Evaporation Models for Estimating Reference Crop Evapotranspiration during Recalibration at Local Conditions

Babakos

Papamichail

Tziachris³

et al. 2020

Hydrology

View full text Add to dashboard Cite

A classic method for assessing the reference crop evapotranspiration (ETo) is the pan evaporation (Epan) method that uses Epan measurements and pan coefficient (kp) models, which can be functions of relative humidity (RH), wind speed (u2), and temperature (T). The aim of this study is to present a methodology for evaluating the robustness of regression coefficients associated to climate parameters (RH, u2, and T) in pan method models during recalibration at local conditions. Two years of daily data from April to October (warm season) of meteorological parameters, Epan measurements from class A pan evaporimeter and ETo estimated by ASCE-standardized method for the climatic conditions of Thessaloniki (Greece, semi-arid environment), were used. The regression coefficients of six general nonlinear (NLR) regression Epan models were analyzed through recalibration using a technique called “random cross-validation nonlinear regression RCV-NLR” that produced 1000 random splits of the initial dataset into calibration and validation sets using a constant proportion (70% and 30%, respectively). The variance of the regression coefficients was analyzed based on the 95% interval of the highest posterior density distribution. NLR models that included coefficients with a 95% HPD interval that fluctuates in both positive and negative values were considered nonrobust. The machine-learning technique of random forests (RF) was also used to build a RF model that includes Epan, u2, RH, and T parameters. This model was used as a benchmark for evaluating the predictive accuracy of NLR models but, also, for assessing the relative importance of the predictor climate variables if they were all included in one NLR model. The findings of this study indicated that locally calibrated NLR functions that use only the Epan parameter presented better results, while the inclusion of additional climate parameters was redundant and led to underfitting.

show abstract

A ranking system for comparing models' performance combining multiple statistical criteria and scenarios: The case of reference evapotranspiration models

Cited by 15 publications

References 59 publications

Modulation of the goodness of fit in hydrological modelling based on inner balance errors

Modulation of the goodness of fit in hydrological modelling based on inner balance errors

Combining biophysical parameters with thermal and RGB indices using machine learning models for predicting yield in yellow rust affected wheat crop

Assessing the Robustness of Pan Evaporation Models for Estimating Reference Crop Evapotranspiration during Recalibration at Local Conditions

Contact Info

Product

Resources

About