Online Prediction Under Model Uncertainty via Dynamic Model Averaging: Application to a Cold Rolling Mill

Abstract

We consider the problem of online prediction when it is uncertain what the best prediction model to use is. We develop a method called Dynamic Model Averaging (DMA) in which a state space model for the parameters of each model is combined with a Markov chain model for the correct model. This allows the "correct" model to vary over time. The state space and Markov chain models are both specified in terms of forgetting, leading to a highly parsimonious representation. As a special case, when the model and parameters do not change, DMA is a recursive implementation of standard Bayesian model averaging, which we call recursive model averaging. The method is applied to the problem of predicting the output strip thickness for a cold rolling mill, where the output is measured with a time delay. We found that when only a small number of physically motivated models were considered and one was clearly best, the method quickly converged to the best model, and the cost of model uncertainty was small; indeed DMA performed slightly better than the best physical model. When model uncertainty and the number of models considered were large, our method ensured that the penalty for model uncertainty was small. At the beginning of the process, when control is most difficult, we found that DMA over a large model space led to better predictions than the single best performing physically motivated model. We also applied the method to several simulated examples, and found that it recovered both constant and time-varying regression parameters and model specifications quite well.

A New Index of Financial Conditions

“…In particular, we want a methodology which can choose di¤erent …nancial variables at di¤erent points in time and weight them di¤erently. We develop DMS and DMA methods, adapted from Raftery et al (2010), to achieve this aim.…”

“…In a model averaging exercise, we want to allow for the weights used in the averaging process to change over time, thus leading to dynamic model averaging (DMA). In this paper, we do DMA and DMS using an approach developed in Raftery et al (2010) in an application involving many TVP regression models. The reader is referred to Raftery et al (2010) for a complete derivation and motivation of DMA.…”

“…DMA arises if model averaging is done in period t using tjt 1;j for j = 1; ::; J as weights. The contribution of Raftery et al (2010) is to develop a fast recursive algorithm for calculating tjt 1;j .…”

“…However, we do not work with a single FAVAR, but rather work with a large set of FAVARs which di¤er in which …nancial variables are included. Faced with a large model space and the desire to allow for model change, we use dynamic model averaging (DMA) and selection (DMS) methods developed in Raftery et al (2010). DMS chooses di¤erent …nancial variables to make up the FCI at di¤erent points in time.…”

Yield curve forecast combinations based on bond portfolio performance

“…To this end, it is fundamental to incorporate in the analysis the decision-making process in which forecasts are to be used, since this environment will define how the investor ultimately evaluates the performance of alternative candidate models. This paper builds on the purely statistical dynamic model averaging (hereafter DMA) procedure developed by Raftery, Kárnỳ , and Ettler (2010), to develop an economically motivated algorithm to combine or to select forecasts, such that combination weights reflect the portfolio returns obtained by each individual model. Raftery et al (2010) use a Markov chain to model the problem of real-time prediction under uncertainty regarding the best forecasting model to use.…”