DeepScalper

Sun, Shuo; Xue, Wanqi; Wang, Rundong; He, Xu; Zhu, Junlei; Li, Jian; An, Bo

doi:10.1145/3511808.3557283

Cited by 8 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many deep reinforcement learning methods for quantitative trading have been proposed. DeepScalper (Sun et al 2022) uses a hindsight bonus and auxiliary task to improve the model's generalization ability in intraday trading. DRA (Briola et al 2021) uses LSTM and PPO.…”

Section: Rl For Quantitative Tradingmentioning

confidence: 99%

“…It has been widely used in Cryptocurrency (Crypto) market due to Crypto's 24/7 non-stop trading time, which prevents Crypto holders from overnight risk, and dramatic price fluctuations, which provides more profitable trading opportunities for HFT. Although reinforcement learning (RL) algorithms (Sun et al 2022;Théate and Ernst 2021;Cumming, Alrajeh, and Dickens 2015) have achieved outstanding results in low-frequency trading in traditional financial markets like stock or futures, few maintain robust performance under the setting of HFT due to two challenges:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading

Qin,

Sun,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

High-frequency trading (HFT) is using computer algorithms to make trading decisions in short time scales (e.g., second-level), which is widely used in the Cryptocurrency (Crypto) market, (e.g., Bitcoin). Reinforcement learning (RL) in financial research has shown stellar performance on many quantitative trading tasks. However, most methods focus on low-frequency trading, e.g., day-level, which cannot be directly applied to HFT because of two challenges. First, RL for HFT involves dealing with extremely long trajectories (e.g., 2.4 million steps per month), which is hard to optimize and evaluate. Second, the dramatic price fluctuations and market trend changes of Crypto make existing algorithms fail to maintain satisfactory performances. To tackle these challenges, we propose an Efficient hieArchical Reinforcement learNing method for High Frequency Trading (EarnHFT), a novel three-stage hierarchical RL framework for HFT. In stage I, we compute a Q-teacher, i.e., the optimal action value based on dynamic programming, for enhancing the performance and training efficiency of second level RL agents. In stage II, we construct a pool of diverse RL agents for different market trends, distinguished by return rates, where hundreds of RL agents are trained with different preferences of return rates and only a tiny fraction of them will be selected into the pool based on their profitability. In stage III, we train a minute-level router which dynamically picks a second-level agent from the pool to achieve stable performance across different markets. Through extensive experiments in various market trends on Crypto markets in a high-fidelity simulation trading environment, we demonstrate that EarnHFT significantly outperforms 6 state-of-art baselines in 6 popular financial criteria, exceeding the runner-up by 30% in profitability.

show abstract

Section: Rl For Quantitative Tradingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%