Active learning machine learns to create new quantum experiments

Melnikov, A. A.; Nautrup, Hendrik Poulsen; Krenn, Mario; Dunjko, Vedran; Tiersch, Markus; Zeilinger, Anton; Briegel, Hans J.

doi:10.1073/pnas.1714936115

Cited by 295 publications

(276 citation statements)

References 58 publications

Supporting

Mentioning

262

Contrasting

Unclassified

Order By: Relevance

“…Henceforth, we will focus on (two-layer) PS, due to its simpler update rule (equation (8)) and to investigate the potential of t-PS. Indeed, GridWorld has been already investigated in the context of PS [44], a relevant example being the design of optical experiments, which was shown to be representable as a generalized GridWorld [43]. Furthermore, note that for both SARSA and Q-learning we numerically observed a performance very similar to PS.…”

Section: Testing the Architecturementioning

confidence: 65%

“…PS is a recent, physically-motivated RL model [26], which has already found several applications ranging from robotics [41] and quantum error correction [7] to the study of collective behavior [42] and automated experiment design [43]. Decision-making in PS occurs in a network of clips that constitutes the agent's episodic and compositional memory (ECM) (figure 2(a)).…”

Section: Projective Simulationmentioning

confidence: 99%

See 1 more Smart Citation

Photonic architecture for reinforcement learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in quantum optical experiments appears to be within the reach of near-term technology.OPEN ACCESS RECEIVED promising features as compared to electronic processors [27][28][29][30]. For instance, nanosecond-scale routing and reconfigurability have already been demonstrated [31][32][33], while encoding information in photons enables decision-making at the speed of light, only limited by the generation and detection rates. Moreover, the use of phase-change materials for in-memory information processing [34] promises to enhance the energy efficiency, since their properties can be modified without continuous external intervention [35,36]. Importantly, since the architecture uses single photons, decision-making is fueled by genuine quantum randomness. This feature marks a fundamental departure from pseudorandom number generation in conventional devices. (ii) The second contribution is the development of a specific variant of PS based on binary decision trees (tree-PS, or t-PS for short), which is closely connected to the standard PS and suitable for the implementation on a photonic circuit. Furthermore, we discuss how this variant enables key features of artificial intelligence, namely abstraction and generalization [37,38].The Article is structured as follows. In section 2 we summarize the theoretical framework of RL, exemplified by three common approaches: SARSA, Q-learning, and PS. In section 3 we describe the blueprint for a fully integrated, photonic RL agent. We then numerically investigate its performance within two standard RL tasks and under realistic experimental imperfections in section 4. Finally, in section 5 we discuss promising features of this architecture within the context of t-PS.

show abstract

Section: Testing the Architecturementioning

confidence: 65%

Section: Projective Simulationmentioning

confidence: 99%

Photonic architecture for reinforcement learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…It should also be noted that this indeterministic aspect is a fundamental feature of the model in the same way in which other well-known models in physics, such as the ones referred to at the end of section 3.1, are fundamentally indeterministic. In projective simulation, the indeterminism is not added on top of an 50 Recent developments include applications in robotics (Hangl et al 2016(Hangl et al , 2017a and in the design of novel quantum experiments (Melnikov et al 2018).…”

Section: Projective Simulation: Free Agency Under Indeterminismmentioning

confidence: 99%

A Stochastic Process Model for Free Agency under Indeterminism

Müller

Briegel

2018

Dialectica

Self Cite

View full text Add to dashboard Cite

The aim of this paper is to establish that free agency, which is a capacity of many animals including human beings, is compatible with indeterminism: an indeterministic world allows for the existence of free agency. The question of the compatibility of free agency and indeterminism is less discussed than its mirror image, the question of the compatibility of free agency and determinism. It is, however, of great importance for our self‐conception as free agents in our (arguably) indeterministic world. We begin by explicating the notions of indeterminism and free agency and by clarifying the interrelation of free agency and the human‐specific notion of free will. We then situate our claim of the compatibility of free agency and indeterminism precisely in the landscape of the current debate on freedom and determinism, exposing an unhappy asymmetry in that debate. Then we proceed to make our case by describing the mathematically precise, physically motivated model of projective simulation, which employs indeterminism as a central resource for agency modeling. We argue that an indeterministic process of deliberation modeled by the dynamics of projective simulation can exemplify free agency under indeterminism, thereby establishing our compatibility claim: Free agency can develop and thrive in an indeterministic world.

show abstract

“…The search for optimal control can naturally be formulated as reinforcement learning (RL) [11][12][13][14][15][16][17][18][19], a discipline of machine learning. RL has been used in the context of quantum control [17], to design experiments in quantum optics [20], and to automatically generate sequences of gates and measurements for quantum error correction [16,21,22].…”

Section: Introductionmentioning

confidence: 99%

Improving the dynamics of quantum sensors with reinforcement learning

2020

View full text Add to dashboard Cite

Recently proposed quantum-chaotic sensors achieve quantum enhancements in measurement precision by applying nonlinear control pulses to the dynamics of the quantum sensor while using classical initial states that are easy to prepare. Here, we use the cross-entropy method of reinforcement learning (RL) to optimize the strength and position of control pulses. Compared to the quantumchaotic sensors with periodic control pulses in the presence of superradiant damping, we find that decoherence can be fought even better and measurement precision can be enhanced further by optimizing the control. In some examples, we find enhancements in sensitivity by more than an order of magnitude. By visualizing the evolution of the quantum state, the mechanism exploited by the RL method is identified as a kind of spin-squeezing strategy that is adapted to the superradiant damping.

show abstract

Active learning machine learns to create new quantum experiments

Cited by 295 publications

References 58 publications

Photonic architecture for reinforcement learning

Photonic architecture for reinforcement learning

A Stochastic Process Model for Free Agency under Indeterminism

Improving the dynamics of quantum sensors with reinforcement learning

Contact Info

Product

Resources

About