Root mean square error (RMSE) or mean absolute error (MAE)?

Abstract

Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura (2005) and Willmott et al. (2009) are valid, the proposed avoidance of RMSE in favor of MAE is not the solution. Citing the aforementioned papers, many researchers chose MAE over RMSE to present their model evaluation statistics when presenting or adding the RMSE measures could be more beneficial. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereas Willmott et al. (2009) indicated that the sums-ofsquares-based statistics do not satisfy this rule. In the end, we discussed some circumstances where using the RMSE will be more beneficial. However, we do not contend that the RMSE is superior over the MAE. Instead, a combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.

Assessment of Aboveground Woody Biomass Dynamics Using Terrestrial Laser Scanner and L-Band ALOS PALSAR Data in South African Savanna

“…We used 60% of the field AGB dataset for regression model calibration, and the remaining 40% for validation of the resulting predictions. To assess the linearity, we looked at the model performance in terms of the resultant root mean square error (RMSE) and bias (Equations (5) and (6)) from correlating the TLS metrics with both log (AGB) and normal AGB values [68]. To assess error distribution with AGB, we computed residuals (Equation (7)) as the difference between observed and predicted AGB for every TLS metrics.…”

A Prediction Method to Improve Training Management

“…The mean absolute error (MAE) and accuracy are used to evaluate the results [20] as in Equations (7) and (8) [21] respectively.…”

A comparison of Multiple Non-linear regression and neural network techniques for sea surface salinity estimation in the tropical Atlantic ocean based on satellite data

“…Chai and Draxler (2014) have demonstrated that the RMSE is not ambiguous in its meaning, and is more appropriate to use than the MAE when model errors follow a normal distribution [8].…”

“…• RMSE: As shown in Equation (10), using this quadratic scoring rule, since the errors are squared before they are averaged, the RMSE gives a relatively high weight to large errors. Thus, this rule is most useful when large errors are particularly undesirable [8].…”