Modelling Stock Markets by Multi-agent Reinforcement Learning

Lussange, J.; Lazarevich, Ivan; Bourgeois‐Gironde, Sacha; Palminteri, Stefano; Gutkin, Boris

doi:10.1007/s10614-020-10038-w

Cited by 38 publications

(34 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works in financial and machine learning literature have exploited RL in different financial market studies, e.g., financial signal representation [4,7,14], building algorithmic trading [4,8-10, 15, 16], portfolio management [11,17,18], optimizing trade execution [19], Foreign Exchange (FX) asset allocations [20], changes in market regimes [11], and stock market modelling [21,22]. Building algorithmic trading using RL has been the focus of many studies for a range of market settings.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Algorithmic Trading Strategy Using Reinforcement Learning and Directional Change

Aloud

Alkhamees

2021

IEEE Access

View full text Add to dashboard Cite

Designing a profitable trading strategy plays a critical role in algorithmic trading, where the algorithm can manage and execute automated trading decisions. Determining a specific trading rule for trading at a particular time is a critical research problem in financial market trading. However, an intelligent, and a dynamic algorithmic trading driven by the current patterns of price time-series data may help deal with this issue. Thus, Reinforcement Learning (RL) can achieve optimal dynamic algorithmic trading by considering the price time-series as its environment. A comprehensive representation of the environment states is indeed vital for proposing a dynamic algorithmic trading using RL. Therefore, we propose a representation of the environment states using the Directional Change (DC) event approach with a dynamic DC threshold. We refer to the proposed algorithmic trading approach as the DCRL trading strategy. In addition, the proposed DCRL trading strategy was trained using the Q-learning algorithm to find an optimal trading rule. We evaluated the DCRL trading strategy on real stock market data (S&P500, NASDAQ, and Dow Jones, for five years period from 2015-2020), and the results demonstrate that the DCRL state representation policies obtained more substantial trading returns and improved the Sharpe Ratios in a volatile stock market. A series of performance analyses demonstrate the robust performance and extensive applicability of the proposed DCRL trading strategy. INDEX TERMS Machine learning; reinforcement learning; Q-learning; directional change event; algorithmic trading; stock market.

show abstract

Section: Related Workmentioning

confidence: 99%

“…However, this study's main drawback was the assumption that the quantity of each buy order is significantly high to increase the price of the traded security. The work by [22] designed a next-generation multi-agent systems (MAS) stock market simulator. Each agent learns price forecasting and stock trading autonomously via RL.…”

Section: Related Workmentioning

confidence: 99%

Intelligent Algorithmic Trading Strategy Using Reinforcement Learning and Directional Change

Aloud

Alkhamees

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Lussange et al 31 designed a multi-agent network to reproduce the micro-dynamics of a stock price where each agent learned to trade autonomously via RL. To the best of author's knowledge, there is yet limited or no research which presents detailed numerical analyses on the effects of informational disparity (defined through random bridge processes) and the LOB depth on price discovery in a market of N agents and also incorporates RL to explore the optimal exploitation of long-lived superior information.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement-learning-based optimal trading in a simulated futures market with heterogeneous agents

Aydın

2021

SIMULATION

View full text Add to dashboard Cite

This paper simulates a futures market with multiple agents and sequential auctions, where agents receive long-lived heterogeneous signals on the true value of an asset and with a known deadline. The evolution of the amount of differential information and its impact on the distribution of overall gains and the pace of truth discovery is examined for various depth levels of the limit order book (LOB). The paper also formulates a dynamic programming model for the problem and presents an associated reinforcement learning (RL) algorithm for finding optimal strategy in exploiting informational disparity. This is done from the perspective of an agent whose information is superior to the collective information of the rest of the market. Finally, a numerical analysis is presented based on a futures market example to validate the proposed methodology for finding the optimal strategy. We find evidence in favor of a waiting strategy where agent does not reveal her signal until the last auction before the deadline. This result may help bring more insight into the micro-structural dynamics that work against market efficiency.

show abstract

“…Finance is a particularly challenging playground for deep reinforcement learning (DRL) [60,23], including investigating market fragility [51], developing profitable strategies [38,68,69], and assessing portfolio risk [43,7]. However, establishing near-real market environments and benchmarks on financial reinforcement learning are challenging due to three major factors, namely, low signalto-noise ratio (SNR) of financial data, survivorship bias of historical data, and model overfitting in the backtesting stage.…”

Section: Introductionmentioning

confidence: 99%

FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning

Liu¹,

Xia²,

Rui³

et al. 2022

Preprint

View full text Add to dashboard Cite

Finance is a particularly difficult playground for deep reinforcement learning. However, establishing high-quality market environments and benchmarks for financial reinforcement learning is challenging due to three major factors, namely, low signal-to-noise ratio of financial data, survivorship bias of historical data, and model overfitting in the backtesting stage. In this paper, we present an openly accessible FinRL-Meta library that has been actively maintained by the AI4Finance community. First, following a DataOps paradigm, we will provide hundreds of market environments through an automatic pipeline that collects dynamic datasets from real-world markets and processes them into gym-style market environments. Second, we reproduce popular papers as stepping stones for users to design new trading strategies. We also deploy the library on cloud platforms so that users can visualize their own results and assess the relative performance via community-wise competitions. Third, FinRL-Meta provides tens of Jupyter/Python demos organized into a curriculum and a documentation website to serve the rapidly growing community.

show abstract

Modelling Stock Markets by Multi-agent Reinforcement Learning

Cited by 38 publications

References 86 publications

Intelligent Algorithmic Trading Strategy Using Reinforcement Learning and Directional Change

Intelligent Algorithmic Trading Strategy Using Reinforcement Learning and Directional Change

Reinforcement-learning-based optimal trading in a simulated futures market with heterogeneous agents

FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning

Contact Info

Product

Resources

About