Measuring Forecasting Accuracy: Problems and Recommendations (by the Example of SKU-Level Judgmental Adjustments)

Davydenko, Andrey; Fildes, Robert

doi:10.1007/978-3-642-39869-8_4

Cited by 11 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mean absolute percentage error (MAPE) is not preferred for load forecast problems since load values, especially in ILP, are usually small. This can result in large MAPE values which do not correctly reflect the performance of the forecasting algorithms [34].…”

Section: Simulation Resultsmentioning

confidence: 99%

Federated Learning with Hyperparameter-based Clustering for Electrical Load Forecasting

Gholizadeh¹,

Musílek²

2021

Preprint

View full text Add to dashboard Cite

Electrical load prediction has become an integral part of power system operation. Deep learning models have found popularity for this purpose. However, to achieve a desired prediction accuracy, they require huge amounts of data for training. Sharing electricity consumption data of individual households for load prediction may compromise user privacy and can be expensive in terms of communication resources. Therefore, edge computing methods, such as federated learning, are gaining more importance for this purpose. These methods can take advantage of the data without centrally storing it. This paper evaluates the performance of federated learning for short-term forecasting of individual house loads as well as the aggregate load. It discusses the advantages and disadvantages of this method by comparing it to centralized and local learning schemes. Moreover, a new client clustering method is proposed to reduce the convergence time of federated learning. The results show that federated learning has a good performance with a minimum root mean squared error (RMSE) of 0.117kWh for individual load forecasting.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Federated Learning with Hyperparameter-based Clustering for Electrical Load Forecasting

Gholizadeh¹,

Musílek²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…One approach involves providing practitioners with a statistical forecast of the demand for their product, where such forecasts are often based simply on extrapolations of past sales patterns. Thus, it is common practice, in both the fashion and other industries, for practitioners to apply judgmental adjustments to these forecasts, ostensibly in order to take into account market intelligence and other contextual information that has not been allowed for in the statistical algorithm (Davydenko & Fildes, 2014;Fildes, Goodwin, Lawrence, & Nikolopoulos, 2009). When this contextual information is reliable and relates to events that will have significant effects on sales, such adjustments can improve the forecast accuracy (Sanders & Ritzman, 2001).…”

Section: Forecasting In the Fashion Industrymentioning

confidence: 97%

The influence of product involvement and emotion on short-term product demand forecasting

Belvedere

Goodwin

2017

International Journal of Forecasting

View full text Add to dashboard Cite

a b s t r a c tSales forecasters in industries like fast-fashion face challenges posed by short and highly volatile sales time series. Computers can produce statistical forecasts, but these are often adjusted judgmentally to take into account factors such as market intelligence. We explore the effects of two potential influences on these adjustments: the forecaster's involvement with the product category and their emotional reactions to particular products. Two forecasting experiments were conducted using data from a major Italian leather fashion goods producer. The participants' judgmental adjustments tended to lower the forecast accuracy, but especially when the participants had strong preferences for particular products. This appeared to result from a false consensus effect. The most accurate forecasts were made when the participants had no knowledge of the product and only received time series information, though a high level of involvement with the product category also led to a greater accuracy.

show abstract

“…Similar examples have been discussed in the forecasting literature depicting this asymmetric nature of the metric. For example, Davydenko and Fildes (2016), discussing Kolassa and Schütz’s (2007) paper, provide the following example on the issue:…”

Section: Literature Reviewmentioning

confidence: 99%

“…The fundamental error unit is given by: That is, the forecast Error at time t is the difference between the Forecast at time t and the observed Actual at time t (Ampountolas et al , 2022, p. 83). A multitude of error measures have been proposed, tested and applied over the years (Davydenko and Fildes, 2016). Among them, the mean absolute percentage error (MAPE) is probably the most widely used (see, for example, the discussion in Swanson et al , 2011 and Huang et al , 2022).…”

Section: Introductionmentioning

confidence: 99%

The MSapeMER: a symmetric, scale-free and intuitive forecasting error measure for hospitality revenue management

Schwartz

Webb

2023

IJCHM

View full text Add to dashboard Cite

Purpose Mean absolute percentage error (MAPE) is the primary forecast evaluation metric in hospitality and tourism research; however its main shortcoming is that it is asymmetric. The asymmetry occurs due to over or under forecasts that introduce bias into forecast evaluation. This study aims to explore the nature of asymmetry and designs a new measure, one that reduces the asymmetric properties while maintaining MAPE’s scale-free and intuitive interpretation characteristics. Design/methodology/approach The study proposes and tests a new forecasting accuracy measure for hospitality revenue management (RM). A computer simulation is used to assess and demonstrate the problem of asymmetry when forecasting with MAPE, and the new measures’ (MSapeMER, that is, Mean of Selectively applied Absolute Percentage Error or Magnitude of Error Relative to the estimate) ability to reduce it. The MSapeMER’s effectiveness is empirically validated by using a large set of hotel forecasts. Findings The study demonstrates the ability of the MSapeMER to reduce the asymmetry bias generated by MAPE. Furthermore, this study demonstrates that MSapeMER is more effective than previous attempts to correct for asymmetry bias. The results show via simulation and empirical investigation that the error metric is more stable and less swayed by the presence of over and under forecasts. Research limitations/implications It is recommended that hospitality RM researchers and professionals adopt MSapeMER when using MAPE to evaluate forecasting performance. The MSapeMER removes the potential bias that MAPE invites due to its calculation and presence of over and under forecasts. Therefore, forecasting evaluations may be less affected by the presence of over and under forecasts and their ability to bias forecasting results. Practical implications Hospitality RM should adopt this measure when MAPE is used, to reduce biased decisions driven by the “asymmetry of MAPE.” Originality/value The MAPE error metric exhibits an asymmetry problem, and this paper proposes a more effective solution to reduce biased results with two major methodological contributions. It is first to systematically study the characteristics of MAPE’s asymmetry, while proposing and testing a measure that considerably reduces the amount of asymmetry. This is a critical contribution because MAPE is the primary forecasting metric in hospitality and tourism studies. The second methodological contribution is a procedure developed to “quantify” the asymmetry. The approach is demonstrated and allows future research to compare asymmetric characteristics among various accuracy measures.

show abstract

Measuring Forecasting Accuracy: Problems and Recommendations (by the Example of SKU-Level Judgmental Adjustments)

Cited by 11 publications

References 22 publications

Federated Learning with Hyperparameter-based Clustering for Electrical Load Forecasting

Federated Learning with Hyperparameter-based Clustering for Electrical Load Forecasting

The influence of product involvement and emotion on short-term product demand forecasting

The MSapeMER: a symmetric, scale-free and intuitive forecasting error measure for hospitality revenue management

Contact Info

Product

Resources

About