Referential kNN Regression for Financial Time Series Forecasting

It is useful to predict future values in time series data, for example when there are many sensors monitoring environments such as urban space. The Gaussian Process (GP) model is considered as a promising technique for this setting. However, the GP model requires too high a training cost to be tractable for large data. Though approximate methods have been proposed to improve GP's scalability, they usually can only capture global trends in the data and fail to preserve small-scale patterns, resulting in unsatisfactory performance.We propose a new method to apply the GP for sensor time series prediction. Instead of (eagerly) training GPs on entire datasets, we custom-build query-dependent GPs on small fractions of the data for each prediction request.Implementing this idea in practice at scale requires us to overcome two obstacles. On the one hand, a central challenge with such a semi-lazy learning model is the substantial model-building effort at kNN query time, which could lead to unacceptable latency. We propose a novel two-level inverted-like index to support kNN search using the DTW on the GPU, making such "just-in-time" query-dependent model construction feasible for real-time applications.On the other hand, several parameters should be tuned for each time series individually since different sensors have different data generating processes in diverse environments. Manually configuring the parameters is usually not feasible due to the large number of sensors. To address this, we devise an adaptive auto-tuning mechanism to automatically determine and dynamically adjust the parameters for each time series with little human assistance.Our method has the following strong points: (a) it can make prediction in real time without a training phase; (b) it can yield superior prediction accuracy; and (c) it can effectively estimate the analytical predictive uncertainty.To illustrate our points, we present SMiLer, a semi-lazy time series prediction system for sensors. Extensive experiments on real-world datasets demonstrate its effectivePermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. ness and efficiency. In particular, by devising a two-level inverted-like index on the GPU with an enhanced lower bound of the DTW, SMiLer accelerates the efficiency of kNN search by one order of magnitude over its baselines. The prediction accuracy of SMiLer is better than the state-of-the-art competitors (up to 10 competitors) with better estimation of predictive uncertainty.

show abstract

“…average) over these query results [44,18,11,8]. These methods however cannot estimate the analytical predictive uncertainty directly.…”

Section: Time Series Prediction Methodsmentioning

confidence: 99%

“…(8) and Eqn. (9), λi,j(t) is an effectively exponential smoothing average of the posterior probability of the predictor fi,j over time.…”

Section: Auto-tuning With Self-adaptive Predictionmentioning

confidence: 98%

SMiLer

Zhou

Tung

2015

Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…The machine-learning method kNN was used for time series prediction by various researchers (Troncoso Lora et al, 2004;Imandoust and Bolandraftar, 2013;Ban et al, 2013). The main idea of the kNN technique for pattern classification is based on the similarity of the individuals.…”

Section: Learning Methodsmentioning

confidence: 99%

AME-WPC: Advanced model for efficient workload prediction in the cloud

Cetinski

Jurič

2015

Journal of Network and Computer Applications

View full text Add to dashboard Cite

“…In terms of time series prediction, there has been research on financial forecasting [20] using kNN regression. Parmezan et al [21] propose a modification of the kNN algorithm for time series prediction, whereas Do et al [22] employ a temporal and frequency metric for a k nearest-neighbor classification of time series.…”

Section: Time Series Predictionmentioning

confidence: 99%

Towards a Better Understanding of Public Transportation Traffic: A Case Study of the Washington, DC Metro

Truong¹,

Gkountouna²,

Pfoser³

et al. 2018

Urban Science

View full text Add to dashboard Cite

Abstract:The problem of traffic prediction is paramount in a plethora of applications, ranging from individual trip planning to urban planning. Existing work mainly focuses on traffic prediction on road networks. Yet, public transportation contributes a significant portion to overall human mobility and passenger volume. For example, the Washington, DC metro has on average 600,000 passengers on a weekday. In this work, we address the problem of modeling, classifying and predicting such passenger volume in public transportation systems. We study the case of the Washington, DC metro exploring fare card data, and specifically passenger in-and outflow at stations. To reduce dimensionality of the data, we apply principal component analysis to extract latent features for different stations and for different calendar days. Our unsupervised clustering results demonstrate that these latent features are highly discriminative. They allow us to derive different station types (residential, commercial, and mixed) and to effectively classify and identify the passenger flow of "unknown" stations. Finally, we also show that this classification can be applied to predict the passenger volume at stations. By learning latent features of stations for some time, we are able to predict the flow for the following hours. Extensive experimentation using a baseline neural network and two naïve periodicity approaches shows the considerable accuracy improvement when using the latent feature based approach.

show abstract

Referential kNN Regression for Financial Time Series Forecasting

Cited by 47 publications

References 17 publications

SMiLer

SMiLer

AME-WPC: Advanced model for efficient workload prediction in the cloud

Towards a Better Understanding of Public Transportation Traffic: A Case Study of the Washington, DC Metro

Contact Info

Product

Resources

About