Deep Reinforcement Learning for Trading—A Critical Survey

Millea, Adrian

doi:10.3390/data6110119

Cited by 21 publications

(17 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cryptocurrency market has gained a lot of popularity worldwide, among which Bitcoin, makes the top news almost weekly now (Millea, 2021) .The dataset used for this paper was Yahoo Finance OHLCV raw data ("Bitcoin USD (BTC-USD) Interactive Price Chart -Yahoo Finance," n.d.), along with Google trends data ("Google Trends," n.d.).The training period was from 2015/1/4 to 2019/12/31 and the evaluation period was from 2020/1/1 to 2022/1/31. Meanwhile, in the training section, data from 2014/12/20 to 2015/1/3 have been used to calculate the starting points of moving averages that are needed in the indicators.…”

Section: Train and Evaluation Datamentioning

confidence: 99%

Algorithmic trading on financial time series using Deep Reinforcement Learning

Asghari,

Mozayani

2024

Preprint

View full text Add to dashboard Cite

The use of technology in financial markets has led to extensive changes in conventional trading structures.Today, most orders that reach exchanges are created by algorithmic trading agents. Today, machine learning-based methods play an important role in building automated trading systems. The increasing complexity and dynamism of financial markets are among the key challenges of these methods. The most widely used machine learning approach is supervised learning, but in interactive environments, the use of supervised learning alone has limitations such as difficulty in defining appropriate labels and lack of modeling of the dynamic nature of the market. Due to the good performance of deep reinforcement learning-based approaches, we will use these approaches to solve the mentioned problems. In this paper, we presented a deep reinforcement learning framework for trading in the financial market, a set of input features and indicators selected and tailored to the purpose of the problem, reward function, appropriate models based on fully connected, convolutional and hybrid networks. The proposed top models traded under real market conditions such as transaction costs and then were evaluated. In addition to outperforming the buy and hold strategy, these models achieved excellent cumulative returns while having appropriate risk metrics.

show abstract

Section: Train and Evaluation Datamentioning

confidence: 99%

Algorithmic trading on financial time series using Deep Reinforcement Learning

Asghari,

Mozayani

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Since its inception in 2015 [1], deep reinforcement learning (DRL) has found many applications in a wide set of domains [2]. In recent years, its use in trading has become quite popular, at least in academia (for an overview, see [3] or [4]).…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Millea

2023

Analytics

View full text Add to dashboard Cite

We present a hierarchical reinforcement learning (RL) architecture that employs various low-level agents to act in the trading environment, i.e., the market. The highest-level agent selects from among a group of specialized agents, and then the selected agent decides when to sell or buy a single asset for a period of time. This period can be variable according to a termination function. We hypothesized that, due to different market regimes, more than one single agent is needed when trying to learn from such heterogeneous data, and instead, multiple agents will perform better, with each one specializing in a subset of the data. We use k-meansclustering to partition the data and train each agent with a different cluster. Partitioning the input data also helps model-based RL (MBRL), where models can be heterogeneous. We also add two simple decision-making models to the set of low-level agents, diversifying the pool of available agents, and thus increasing overall behavioral flexibility. We perform multiple experiments showing the strengths of a hierarchical approach and test various prediction models at both levels. We also use a risk-based reward at the high level, which transforms the overall problem into a risk-return optimization. This type of reward shows a significant reduction in risk while minimally reducing profits. Overall, the hierarchical approach shows significant promise, especially when the pool of low-level agents is highly diverse. The usefulness of such a system is clear, especially for human-devised strategies, which could be incorporated in a sound manner into larger, powerful automatic systems.

show abstract

“…Thus, deep reinforcement learning is often used for sequential decision-making tasks performed in a complex environment, such as video games [6], that may consist of multiple variables. In the context of quantitative trading, many previous works used deep reinforcement learning to make practical and profit-maximizing decisions that outperform traditional quantitative trading methods and cutting-edge models by observing raw time series, convolution-processed time series, technical indicators, or correlated pairs of various assets, such as stocks, futures contracts, commodities, foreign exchanges, and even cryptocurrencies [7].…”

Section: Introductionmentioning

confidence: 99%

Generalized Deep Reinforcement Learning for Trading

Sim

Kirk

2023

J Stud Res

View full text Add to dashboard Cite

This paper proposes generalized deep reinforcement learning with multivariate state space, discrete rewards, and adaptive synchronization for trading any stock held in the S&P 500. Specifically, the proposed trading model observes the daily historical data of a stock held in the S&P 500 and multiple market-indicating securities (SPY, IEF, EUR=X, GSG), selects a trading action, and observes a discrete reward that is based on the correctness of the selected action and independent of the volatility of stocks. The proposed trading model’s reward-maximizing behavior is optimized by using a standard deep q-network (DQN) with adaptive synchronization that stabilizes and enables to track learning performance on generalizing new experiences from each stock. The proposed trading model was trained on the top 50 holdings of the S&P 500 and tested on the top 100 holdings of the S&P 500 starting from 2006 to 2022. Experimental results suggest that the proposed trading model significantly outperforms the 100% long-strategy benchmark in terms of annualized return, Sharpe ratio, and maximum drawdown.

show abstract

Deep Reinforcement Learning for Trading—A Critical Survey

Cited by 21 publications

References 55 publications

Algorithmic trading on financial time series using Deep Reinforcement Learning

Algorithmic trading on financial time series using Deep Reinforcement Learning

Hierarchical Model-Based Deep Reinforcement Learning for Single-Asset Trading

Generalized Deep Reinforcement Learning for Trading

Contact Info

Product

Resources

About