Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model

Gasperov, Bruno; Kostanjčar, Zvonko

doi:10.1109/lcsys.2022.3166446

Cited by 6 publications

(3 citation statements)

References 29 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gašperov and Konstanjčar [ 25 ] tackle the problem be means of an ensemble of supervised learning models that provide predictive buy/sell signals as inputs to a DRL network trained with a genetic algorithm. The same authors have recently explored the use of a soft actor-critic RL algorithm in market making, to obtain a continuous action space of spread values [ 28 ]. Comprehensive examinations of the use of RL in market making can be found in Gašperov et al [ 29 ] and Patel [ 30 ].…”

Section: Related Work On Machine Learning In Tradingmentioning

confidence: 99%

“…We chose a discrete action space for our experiment to apply RL to manipulate AS-related parameters, aiming keep the algorithm as simple and quickly trainable as possible. A continuous action space, as the one used to choose spread values in [ 28 ], may possibly perform better, but the algorithm would be more complex and the training time greater.…”

Section: Modelsmentioning

confidence: 99%

See 1 more Smart Citation

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm

Marín

Vera²,

Eduardo³

2022

PLoS ONE

View full text Add to dashboard Cite

Market making is a high-frequency trading problem for which solutions based on reinforcement learning (RL) are being explored increasingly. This paper presents an approach to market making using deep reinforcement learning, with the novelty that, rather than to set the bid and ask prices directly, the neural network output is used to tweak the risk aversion parameter and the output of the Avellaneda-Stoikov procedure to obtain bid and ask prices that minimise inventory risk. Two further contributions are, first, that the initial parameters for the Avellaneda-Stoikov equations are optimised with a genetic algorithm, which parameters are also used to create a baseline Avellaneda-Stoikov agent (Gen-AS); and second, that state-defining features forming the RL agent’s neural network input are selected based on their relative importance by means of a random forest. Two variants of the deep RL model (Alpha-AS-1 and Alpha-AS-2) were backtested on real data (L2 tick data from 30 days of bitcoin–dollar pair trading) alongside the Gen-AS model and two other baselines. The performance of the five models was recorded through four indicators (the Sharpe, Sortino and P&L-to-MAP ratios, and the maximum drawdown). Gen-AS outperformed the two other baseline models on all indicators, and in turn the two Alpha-AS models substantially outperformed Gen-AS on Sharpe, Sortino and P&L-to-MAP. Localised excessive risk-taking by the Alpha-AS models, as reflected in a few heavy dropdowns, is a source of concern for which possible solutions are discussed.

show abstract

Section: Related Work On Machine Learning In Tradingmentioning

confidence: 99%

Section: Modelsmentioning

confidence: 99%

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm

Marín

Vera²,

Eduardo³

2022

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Although the Hawkes model can be used in various fields of finance, such as credit risk analysis (Errais et al, 2010;Aït-Sahalia et al, 2015;Ma and Xu, 2016;Ketelbuters and Hainaut, 2022) and optimal execution (Choi et al, 2021;Da Fonseca and Malevergne, 2021;Jusselin, 2021;Gašperov and Kostanjčar, 2022), studies focusing on price dynamics, such as our study, are also abundant. To mention a few, Bacry et al (2013a) used the multivariate Hawkes model for the tick dynamics of asset prices with the applications of microstructure noise analysis.…”

Section: Introductionmentioning

confidence: 99%

Multi-kernel property in high-frequency price dynamics under Hawkes model

Lee¹

2023

Preprint

View full text Add to dashboard Cite

This study investigates and uses multi-kernel Hawkes models to describe a highfrequency mid-price process. Each kernel represents a different responsive speed of market participants. Using the conditional Hessian, we examine whether the numerical optimizer effectively finds the global maximum of the log-likelihood function under complicated modeling. Empirical studies that use stock prices in the US equity market show the existence of multi-kernels classified as ultra-high-frequency (UHF), very-high-frequency (VHF), and high-frequency (HF). We estimate the conditional expectations of arrival times and the degree of contribution to the high-frequency activities for each kernel.

show abstract