Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach

Abstract

In recent years, considerable efforts have been devoted to developing AI techniques for finance research and applications. For instance, AI techniques (e.g., machine learning) can help traders in quantitative trading (QT) by automating two tasks: market condition recognition and trading strategies execution. However, existing methods in QT face challenges such as representing noisy high-frequent financial data and finding the balance between exploration and exploitation of the trading agent with AI techniques. To address the challenges, we propose an adaptive trading model, namely iRDPG, to automatically develop QT strategies by an intelligent trading agent. Our model is enhanced by deep reinforcement learning (DRL) and imitation learning techniques. Specifically, considering the noisy financial data, we formulate the QT process as a Partially Observable Markov Decision Process (POMDP). Also, we introduce imitation learning to leverage classical trading strategies useful to balance between exploration and exploitation. For better simulation, we train our trading agent in the real financial market using minute-frequent data. Experimental results demonstrate that our model can extract robust market features and be adaptive in different markets.

A Multistage Hybrid Model ICEEMDAN-SE-VMD-RDPG for a Multivariate Solar Irradiance Forecasting

“…Furthermore, the implementation of the RDPG model for different applications, which require a more rapid and accurate response compared to the forecasting, verifies the suitability of the RDPG. Consequently, the RDPG was applied to control traffic lights in transportation [73], learn adaptive behavior in driving [74], perform adaptive trading for different markets [75], and to reduce errors of robot positions and joint torques [42]. These applications of the RDPG algorithm signify that the RDPG based ISVR model is feasible for both offline and real-time applications in any type of complex environment.…”

Time Your Hedge With Deep Reinforcement Learning

“…Secondly, open high low prices tend to be highly correlated creating some noise in the inputs. Third, the concept of volatility is crucial to detect regime change and is surprisingly absent from these works as well as from other works like (Yu et al 2019;Wang and Zhou 2019;Liu et al 2020;Ye et al 2020;Li et al 2019;Xiong et al 2019).…”

“…Third, there is no consideration of online learning to adapt to changing environment as well as the incorporation of transaction costs. A second stream of research around deep reinforcement learning has emerged to address these points (Jiang and Liang 2016;Jiang, Xu, and Liang 2017;Liang et al 2018;Yu et al 2019;Wang and Zhou 2019;Liu et al 2020;Ye et al 2020;Li et al 2019;Xiong et al 2019;Benhamou et al 2020a;2020b). The dynamic nature of reinforcement learning makes it an obvious candidate for changing environment (Jiang and Liang 2016;Jiang, Xu, and Liang 2017;Liang et al 2018).…”

“…The dynamic nature of reinforcement learning makes it an obvious candidate for changing environment (Jiang and Liang 2016;Jiang, Xu, and Liang 2017;Liang et al 2018). Transaction costs can be easily included in rules (Liang et al 2018;Yu et al 2019;Wang and Zhou 2019;Liu et al 2020;Ye et al 2020;Yu et al 2019). However, these works, except (Ye et al 2020) and (Benhamou et al 2020a) rely only on time series of open high low close prices, which are known to be very noisy.…”

“…It is not surprising that a delayed action (with one period lag) after observation makes the learning process more challenging for the DRL agent as influence of variables tends to decrease with time. Surprisingly, this salient modeling characteristic is ignored in existing literature (Jiang, Xu, and Liang 2017;Liang et al 2018;Yu et al 2019;Wang and Zhou 2019;Liu et al 2020;Ye et al 2020;Li et al 2019).…”