Temporal Attention-Augmented Bilinear Network for Financial Time-Series Data Analysis

Tran, Dat Thanh; Iosifidis, Alexandros; Kanniainen, Juho; Gabbouj, Moncef

doi:10.1109/tnnls.2018.2869225

Cited by 190 publications

(143 citation statements)

References 63 publications

(91 reference statements)

Supporting

Mentioning

117

Contrasting

Unclassified

Order By: Relevance

“…Besides NLP and computer vision [16]- [18], attentive models have been successfully adopted in many other different fields, such as speech recognition [19]- [21], This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ recommendation [22], [23], time-series analysis [24], [25], games [26], and mathematical problems [27], [28]. In NLP, after an initial exploration by a number of seminal papers [2], [59], a fast-paced development of new attention models and attentive architectures ensued, resulting in a highly diversified architectural landscape.…”

Section: Introductionmentioning

confidence: 99%

Attention in Natural Language Processing

Galassi

Lippi

Torroni

2021

IEEE Trans. Neural Netw. Learning Syst.

407

130

View full text Add to dashboard Cite

Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.

show abstract

Section: Introductionmentioning

confidence: 99%

Attention in Natural Language Processing

Galassi

Lippi

Torroni

2021

IEEE Trans. Neural Netw. Learning Syst.

407

130

View full text Add to dashboard Cite

show abstract

“…There are several recently proposed machine learning methods for predicting various aspects of the financial markets using limit order book data [11], [15], [20]- [24]. In [15], a set of hand-crafted features are extracted and then a Support Vector Machine (SVM) classifier is utilized to predict whether the mid price of a stock stays stationary, increases or decreases, while a similar task is tackled using a deep learning approach in [11], [20], [24], [25]. A different methodology is used in [21], and [22], where reinforcement learning is used to learn the optimal way to perform trading.…”

Section: Related Workmentioning

confidence: 99%

Temporal Bag-of-Features Learning for Predicting Mid Price Movements Using High Frequency Limit Order Book Data

Passalis

Tefas

Kanniainen

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

Self Cite

View full text Add to dashboard Cite

Time-series forecasting has various applications in a wide range of domains, e.g., forecasting stock markets using limit order book data. Limit order book data provide much richer information about the behavior of stocks than its price alone, but also bear several challenges, such as dealing with multiple price depths and processing very large amounts of data of high-dimensionality, velocity and variety. A well-known approach for efficiently handling large amounts of high-dimensional data is the Bag-of-Features (BoF) model. However, the BoF method was designed to handle multimedia data, such as images. In this work, a novel temporalaware neural BoF model is proposed tailored to the needs of time-series forecasting using high frequency limit order book data. Two separate sets of Radial Basis Function (RBF) and accumulation layers are used in the Temporal Bag-of-Features to capture both the short-term behavior and the long-term dynamics of time-series. This allows for modeling complex temporal phenomena that occur in time-series data and further increase the forecasting ability of the model. Any other neural layer, such as feature transformation layers, or classifiers, such as Multilayer Perceptrons, can be combined with the proposed deep learning approach, which can be trained end-to-end using the back-propagation algorithm. The effectiveness of the proposed method is validated using a large-scale limit order book dataset, containing over 4.5 million limit orders, and it is demonstrated that it greatly outperforms all the other evaluated methods.

show abstract

“…1 , 2 , and 3 denote the position adjustments with respect to α, β, and δ, respectively, while , +1 , , +1 and , +1 are obtained using (1)- (3).…”

Section: A the Proposed Gwo Variantmentioning

confidence: 99%

“…A time series is a sequence of data measured chronologically at a uniform time interval [1]. Time series measurements are prevalent in various domains, such as weather forecast [2], financial market prediction [3], physiological assessment [4] and video analysis [5]. Over the last several decades, many efforts have been made to develop effective time series forecasting models, which can be broadly classified into three categories: 1) statistical models, e.g.…”

Section: Introductionmentioning

confidence: 99%

Evolving CNN-LSTM Models for Time Series Prediction Using Enhanced Grey Wolf Optimizer

2020

View full text Add to dashboard Cite

In this research, we propose an enhanced Grey Wolf Optimizer (GWO) for designing the evolving Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) networks for time series analysis. To overcome the probability of stagnation at local optima and a slow convergence rate of the classical GWO algorithm, the newly proposed variant incorporates four distinctive search mechanisms. They comprise a nonlinear exploration scheme for dynamic search territory adjustment, a chaotic leadership dispatching strategy among the dominant wolves, a rectified spiral local exploitation action, as well as probability distribution-based leader enhancement. The evolving CNN-LSTM models are subsequently devised using the proposed GWO variant, where the network topology and learning hyperparameters are optimized for time series prediction and classification tasks. Evaluated using a number of benchmark problems, the proposed GWO-optimized CNN-LSTM models produce statistically significant results over those from several classical search methods and advanced GWO and Particle Swarm Optimization variants. Comparing with the baseline methods, the CNN-LSTM networks devised by the proposed GWO variant offer better representational capacities to not only capture the vital feature interactions, but also encapsulate the sophisticated dependencies in complex temporal contexts for undertaking time-series tasks.

show abstract

Temporal Attention-Augmented Bilinear Network for Financial Time-Series Data Analysis

Cited by 190 publications

References 63 publications

Attention in Natural Language Processing

Attention in Natural Language Processing

Temporal Bag-of-Features Learning for Predicting Mid Price Movements Using High Frequency Limit Order Book Data

Evolving CNN-LSTM Models for Time Series Prediction Using Enhanced Grey Wolf Optimizer

Contact Info

Product

Resources

About