Reinforcement learning decoders for fault-tolerant quantum computation

Abstract

Topological error correcting codes, and particularly the surface code, currently provide the most feasible road-map towards large-scale fault-tolerant quantum computation. As such, obtaining fast and flexible decoding algorithms for these codes, within the experimentally realistic and challenging context of faulty syndrome measurements, without requiring any final read-out of the physical qubits, is of critical importance. In this work, we show that the problem of decoding such codes can be naturally reformulated as a process of repeated interactions between a decoding agent and a code environment, to which the machinery of reinforcement learning can be applied to obtain decoding agents. While in principle this framework can be instantiated with environments modelling circuit level noise, we take a first step towards this goal by using deepQ learning to obtain decoding agents for a variety of simplified phenomenological noise models, which yield faulty syndrome measurements without including the propagation of errors which arise in full circuit level noise models.

A NEAT quantum error decoder

“…Our results were obtained without the use of spatial information that comes with using convolutional neural networks as in Refs. [13][14][15][16]. We remark that accessing larger code distances still becomes increasingly difficult due to slow convergence, and the genome transplantation procedure was crucial in particular for depolarizing noise.…”

“…Bitflip ∼ 500000 ∼ 1200000 [15] Depolarizing ∼ 900000 ∼ 9000000 [16] Bitflip ∼ 640000 ∼ 1700000 ∼ 3200000 that Ref. [13] deals with faulty measurements, which is a considerably harder decoding task.…”

“…Compared to hard-coded decoding algorithms for stabilizer codes, these machine learning based decoders are more like maximum-likelihood decoders [10,11] than, for instance, the minimum weight perfect matching (MWPM) algorithm that looks for the lowest energy correction [12]. In the RL based approach, the decoding problem is formulated as a movebased single player game [13][14][15][16]: the player proposes a local correction, receives back the new state of the code and wins whenever the error-free state of the code is restored correctly. These machine learning decoders are mostly limited to small sizes (albeit comparable to a possible realistic experimental implementation), though scalability through hybrid approaches is a promising research direction [17].…”

Reinforcement Learning with Neural Networks for Quantum Multiple Hypothesis Testing

“…While RLNN was introduced more than 20 years ago [11,12], interest in these methods was recently rekindled by its remarkable success for Atari games [13,14]. RLNN and other machine-learning approaches have been successfully applied to a variety of problems in quantum information theory: generating error-correcting sequences [15,16], preparation of special quantum states [17][18][19], setting up experimental Bell tests [20], quantum communication [21], fault-tolerant quantum computation [22], quantum control [23][24][25][26], and nonequilibrium quantum thermodynamics [27]. Additionally, RLNN has been applied in the closely related topic of adaptive quantum metrology [28][29][30][31].…”

Quantum compiling by deep reinforcement learning

“…Recently, deep learning has been successfully applied to physics [18][19][20][21] , where unprecedented advancements have been achieved by combining reinforcement learning 22 with deep neural networks into deep reinforcement learning (DRL). DRL, thanks to its ability to identify strategies for achieving a goal in complex configuration spaces without prior knowledge of the system [23][24][25][26][27][28] , has recently been proposed for the control of quantum systems 15,18,[29][30][31][32][33] . In this context, some of us previously applied deep reinforcement learning to control and initialize qubits by continuous pulse sequences 34,35 for coherent transport by adiabatic passage (CTAP) 36 and by digital pulse sequences for stimulated Raman passage (STIRAP) 37,38 , respectively.…”