Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning

Abstract

This paper applies a deep reinforcement learning approach to revisit the hedging problem of variable annuities. Instead of assuming actuarial and financial dualmarket model a priori, the reinforcement learning agent learns how to hedge by collecting anchor-hedging reward signals through interactions with the market. By the recently advanced proximal policy optimization, the pseudo-model-free reinforcement learning agent performs equally well as the correct Delta, while outperforms the misspecified Deltas. The reinforcement learning agent is also integrated with online learning to demonstrate its full adaptive capability to the market.

Deep Equal Risk Pricing of Financial Derivatives with Non-Translation Invariant Risk Measures

“…For this reason, it was used in multiple other works on derivatives pricing and hedging. Various techniques were considered, such as Q-learning by Halperin (2020) and Cao et al (2021), proximal policy optimization by Chong et al (2021), least squares policy iteration and fitted Q-iteration for American option pricing by Li et al (2009), or batch policy gradient by Buehler et al (2019). Moreover, various other financial problems were tackled through reinforcement learning procedures in the literature, for instance, portfolio management by Moody and Wu (1997), Jiang et al (2017), Pendharkar and Cusatis (2018), García-Galicia et al (2019), Wang and Zhou (2020), Ye et al (2020) and Betancourt and Chen (2021); optimal liquidation by Bao and Liu (2019); or trading optimization by Hendricks and Wilcox (2014), Lu (2017) and Ning et al (2021).…”

Deep equal risk pricing of financial derivatives with non-translation invariant risk measures

“…For this reason, it was used in multiple other works on derivatives pricing and hedging. Various techniques were considered such as Q-learning in Halperin (2020) and Cao et al (2021), proximal policy optimization in Chong et al (2021), least squares policy iteration and fitted Q-iteration for American option pricing in Li et al (2009), or batch policy gradient in Buehler et al (2019). Moreover, various other financial problems were tackled through reinforcement learning procedures in the literature, for instance portfolio management as in Moody and Wu (1997), Jiang et al (2017), Pendharkar and Cusatis (2018), García-Galicia et al (2019), Wang and Zhou (2020), Ye et al (2020) and Betancourt and Chen (2021), optimal liquidation, see Bao and Liu (2019), or trading optimization as in Hendricks and Wilcox (2014), Lu (2017) and Ning et al (2018).…”

Premium control with reinforcement learning

“…Carbonneau (2021) uses the methodology in Buehler et al (2019) and studies approaches to risk management of long-term financial derivatives motivated by guarantees and options embedded in life-insurance products. Another approach to deep hedging based on reinforcement learning for managing risks stemming from long-term life-insurance products is presented in Chong et al (2021). Dynamic pricing has been studied extensively in the operations research literature.…”