Exploration vs Exploitation with Partially Observable Gaussian Autoregressive Arms

Kühn, Julia; Mandjes, Michel; Nazarathy, Yoni

doi:10.4108/icst.valuetools.2014.258207

Cited by 1 publication

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our goal is not to merely present a descriptive collection of the related work and gaps in these two lines of researnd for our future work on RL based methods for optimizing locality and globality in probabilistic forecasting. Albeit there is extensive literature work on proposing global methods over local ones or the other way around (de Bezenac et al, 2020;Rangapuram et al, 2018;Kuhn et al, 2015;Rasul et al, 2021;Salinas et al, 2020;Tang et al, 2020;Yao et al, 2017;Zhou et al, 2020;Zhu & Laptev, 2017), the research on hybrid models (global-local, local-global) is still incipient and not always suitable to be applied for collections of multivariate TS, which we mainly take as our focus scope (Hwang et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Sousa

Henriques

2023

Preprint

View full text Add to dashboard Cite

In the current context of big data, the operational forecasting problems are more and more frequently involving the prediction of collections of multivariate, high-dimensional, related time series. The challenge of forecasting groups of time series can be tackled by fitting a single function to all series (global approach) or assuming each series to be a separate prediction problem and fitting one function to each problem (local approach). Although some global models show promising results against local benchmarks, there is still unclarity on how to best control generalization and structural assumptions by calibrating the exploitation of global patterns with local components. This is heavily reliant on the level of complexity from the dependency relationships exhibited between the series within a collection (e.g. indirect latent, local covariate relationships or noise effects) and other dimensions, such as the heterogeneity of the series and its length. For these reasons, there is an acknowledged need to invest further in developing scalable and data-driven hybrid models. The literature proposes Deep Learning (DL) based models built from different data generating processes (such as Auto-Regressive) which combine global models with classical local models (global-local frameworks). These methods frequently capture globality through factorization models or latent deep components and locality via classical local models or networks. These methods present non-linear modelling capabilities but are still either limited by solid theoretical assumptions (e.g. linearity of the underlying data) or predominantly one-dimensional, i.e., the future predictions for a single dimension mainly depend on past values from that same dimension. Simultaneously, dynamic Reinforcement Learning (RL) based models have been widely explored to optimize the balance of global and local signals in general regression problems, frequently through Q-learning algorithms. These derive from the equilibrium of cooperative strategies or the societal value in multi-agent learning systems. Similar models have been successfully applied to time series forecasting problems (such as stock market predictions), addressing the risk of method generalization under varying conditions. This paper conducted a concise literature review focused on these two research streams (global-local frameworks and RL based models) to optimize the balance between globality and locality in forecasting collections of time series. It focuses on their evolution across time and hints at opportunities to close some of the research gaps by intersecting both propositions. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and achieved a selection of 118 publications since 2000. The main findings revealed that global models have achieved strong expressiveness in capturing the most complex structural patterns while still enabling probabilistic outcomes to be delivered through uncertainty estimates. On the other hand, RL based methods depict great benefits in mitigating the risks of generalization by imprinting contextual diversity when predicting each step ahead for each series. Within those, the adoption of other computational learning or evolutionary-based methods (e.g. Genetic Programming) to improve the parametrization of the learning policies is also highlighted as an area of future work yet to be uncovered.

show abstract