Optimizing Quantum Error Correction Codes with Reinforcement Learning

Abstract

Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.

Photonic architecture for reinforcement learning

“…This model does not only exhibit some key features of artificial intelligence, namely generalization and abstraction, but can also boost its learning performance via autonomous defragmentation of its memory. Indeed, both numerical and analytical results suggest that t-PS performs at least as well as the simulated standard PS model, which has already found various applications [7,[41][42][43]. Eventually, we envisage the experimental realization of a photonic RL agent which successfully exploits all these features within a quantum optical environment.…”

“…PS is a recent, physically-motivated RL model [26], which has already found several applications ranging from robotics [41] and quantum error correction [7] to the study of collective behavior [42] and automated experiment design [43]. Decision-making in PS occurs in a network of clips that constitutes the agent's episodic and compositional memory (ECM) (figure 2(a)).…”

“…Modern computing devices are rapidly evolving from handy resources to autonomous machines [1]. On the brink of this new technological revolution [2], reinforcement learning (RL) has emerged as a powerful and flexible tool to enable problem solving at an unprecedented scale, both in computer science [3-63-6] and in physics research [7][8][9][10][11][12][13]. This breakthrough development was in part spurred by the technological achievements of the last decades, which unlocked vast amounts of data and computational power.…”

Machine Learning for Long-Distance Quantum Communication

“…Next, by building a bridge between knowledge about quantum algorithms and actual near-term experimental capabilities, ML can be used to identify problems in which a quantum advantage over a classical approach can be obtained [33][34][35]. Then, ML can be used to realize such algorithms and protocols in quantum devices, by autonomously learning how to control [36][37][38], error-correct [39][40][41][42], and measure [43] quantum devices. Finally, given experimental data, ML can reconstruct quantum states of physical systems [44][45][46], learn a compact representation of these states, and characterize them [47][48][49].…”

“…First, the PS agent has been shown to perform well on problems that, from a RL perspective, are conceptually similar to designing communication networks. In problems that can be mapped to a navigation problem [66], such as the design of quantum experiments [24] and the optimization of quantum error-correction codes [40], PS outperformed methods that were used practically for those problems (and were not based on machine learning). In standard navigation problems, such as the grid-world and mountain-car problems, the basic PS agent shows performance qualitatively and quantitatively similar to the standard tabular RL models of SARSA and Q-learning [66].…”