Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder—DeepBreath

Soleymani, Farzan; Paquet, Éric

doi:10.1016/j.eswa.2020.113456

Cited by 93 publications

(49 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Buying and selling assets result in spending and gaining cash (Soleymani & Paquet, 2020), which implies that:…”

Section: Mathematical Modelmentioning

confidence: 99%

“…Each asset is characterized by a feature vector containing twelve (12) features including opening, closing, low and high prices in addition to financial indicators such as average true range that evaluate the market volatility over a certain period (Soleymani & Paquet, 2020) as illustrated in Table 1.…”

Section: Feature Normalizationmentioning

confidence: 99%

“…Nonetheless, this exploration is time-consuming as it requires a large number of training epochs. In the DeepBreath framework, a SARSA algorithm was employed in order to learn the optimal policy (Soleymani & Paquet, 2020). However, SARSA may lead to abrupt changes in the policy which are not necessarily suitable for portfolio optimization (Celikyurt & Özekici, 2007) as a result of an erratic change in the policy if the latter performs too poorly (Andrew, 1999).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Soleymani,

Paquet

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Portfolio management aims at maximizing the return on investment while minimizing risk by continuously reallocating the assets forming the portfolio. These assets are not independent but correlated during a short time period. A graph convolutional reinforcement learning framework called DeepPocket is proposed whose objective is to exploit the time-varying interrelations between financial instruments. These interrelations are represented by a graph whose nodes correspond to the financial instruments while the edges correspond to a pair-wise correlation function in between assets. DeepPocket consists of a restricted, stacked autoencoder for feature extraction, a convolutional network to collect underlying local information shared among financial instruments and an actor-critic reinforcement learning agent. The actor-critic structure contains two convolutional networks in which the actor learns and enforces an investment policy which is, in turn, evaluated by the critic in order to determine the best course of action by constantly reallocating the various portfolio assets to optimize the expected return on investment. The agent is initially trained offline with online stochastic batching on historical data. As new data become available, it is trained online with a passive concept drift approach to handle unexpected changes in their distributions. DeepPocket is evaluated against five real-life datasets over three distinct investment periods, including during the Covid-19 crisis, and clearly outperformed market indexes.

show abstract

“…Buying and selling assets result in spending and gaining cash (Soleymani & Paquet, 2020), which implies that:…”

Section: Mathematical Modelmentioning

confidence: 99%

Section: Feature Normalizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Soleymani,

Paquet

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…From the same reason, we won't mention papers that try to combine meta-heuristic algorithms (like Genetic Algorithms -GA [3] or Particle Swarm Optimization -PSO [4]) with Reinforcement learning (RL) or multi-agent solutions to this problem. We also rule out the portfolio management part [5] because it is entirely a different system that should be taken into consideration separately when constructing a trading engine [6,7]. We focus on the actual agent -learner that makes the trades, the allocation of funds is not a priority to review in this paper.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review

Pricope

2021

Preprint

View full text Add to dashboard Cite

Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return -profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock trading. Many of the reviewed studies had only proof-of-concept ideals with experiments conducted in unrealistic settings and no real-time trading applications. For the majority of the works, despite all showing statistically significant improvements in performance compared to established baseline strategies, no decent profitability level was obtained. Furthermore, there is a lack of experimental testing in real-time, online trading platforms and a lack of meaningful comparisons between agents built on different types of DRL or human traders. We conclude that DRL in stock trading has showed huge applicability potential rivalling professional traders under strong assumptions, but the research is still in the very early stages of development.

show abstract

“…RL is reported to solve problem statements of financial industry, such as pricing strategy optimization in insurance industry [29], bank marketing campaigns offering credit card services [30], and portfolio managements [31]. RL has been utilised for trading of financial assets on the stock and foreign exchange market.…”

Section: Introductionmentioning

confidence: 99%

Dynamic portfolio rebalancing through reinforcement learning

Lim

Cao

Quek

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Portfolio managements in financial markets involve risk management strategies and opportunistic responses to individual trading behaviours. Optimal portfolios constructed aim to have a minimal risk with highest accompanying investment returns, regardless of market conditions. This paper focuses on providing an alternative view in maximising portfolio returns using Reinforcement Learning (RL) by considering dynamic risks appropriate to market conditions through dynamic portfolio rebalancing. The proposed algorithm is able to improve portfolio management by introducing the dynamic rebalancing of portfolios with vigorous risk through an RL agent. This is done while accounting for market conditions, asset diversifications, risk and returns in the global financial market. Studies have been performed in this paper to explore four types of methods with variations in fully portfolio rebalancing and gradual portfolio rebalancing, which combine with and without the use of the Long Short-Term Memory (LSTM) model to predict stock prices for adjusting the technical indicator centring. Performances of the four methods have been evaluated and compared using three constructed financial portfolios, including one portfolio with global market index assets with different risk levels, and two portfolios with uncorrelated stock assets from different sectors and risk levels. Observed from the experiment results, the proposed RL agent for gradual portfolio rebalancing with the LSTM model on price prediction outperforms the other three methods, as well as returns of individual assets in these three portfolios. The improvements of the returns using the RL agent for gradual rebalancing with prediction model are achieved at about 27.9–93.4% over those of the full rebalancing without prediction model. It has demonstrated the ability to dynamically adjust portfolio compositions according to the market trends, risks and returns of the global indices and stock assets.

show abstract

Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder—DeepBreath

Cited by 93 publications

References 29 publications

Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review

Dynamic portfolio rebalancing through reinforcement learning

Contact Info

Product

Resources

About