Meta-learning within Projective Simulation

Makmal, Adi; Melnikov, A. A.; Dunjko, Vedran; Briegel, Hans J.

doi:10.1109/access.2016.2556579

Cited by 32 publications

(38 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general this should, and can, be automatized. In recent work, it was shown how these parameters can be learned in the context of the PS, but similar results have been obtained for other models as well (see [17], and references therein). One way to think about this setting is to think of each configuration of parameter settings as fixing some particular learning agent/model A k .…”

Section: Explorationsupporting

confidence: 64%

“…Often, we are facing models at the opposite extreme where the parameter space has plenty of structure. For example, in [17], the meta-parameter γ -the "forgetting" parameter, but the details are not relevant here -of the PS model is, is optimized. The analysis of this work also suggests that, in many environments, the abstract mapping, which maps the parameter value to and averaged performance [0, 1] eval → R + , is an unimodal function -it is increasing to some value γ opt , and decreasing afterwards.…”

Section: Explorationmentioning

confidence: 99%

See 1 more Smart Citation

Advances in quantum reinforcement learning

Dunjko

Taylor

Briegel

2017

2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Self Cite

View full text Add to dashboard Cite

In recent times, there has been much interest in quantum enhancements of machine learning, specifically in the context of data mining and analysis. Reinforcement learning, an interactive form of learning, is, in turn, vital in artificial intelligence-type applications. Also in this case, quantum mechanics was shown to be useful, in certain instances. Here, we elucidate these results, and show that quantum enhancements can be achieved in a new setting: the setting of learning models which learn how to improve themselves -that is, those that metalearn. While not all learning models meta-learn, all non-trivial models have the potential of being "lifted", enhanced, to metalearning models. Our results show that also such models can be quantum-enhanced to make even better learners. In parallel, we address one of the bottlenecks of current quantum reinforcement learning approaches: the need for so-called oracularized variants of task environments. Here we elaborate on a method which realizes these variants, with minimal changes in the setting, and with no corruption of the operative specification of the environments. This result may be important in near-term experimental demonstrations of quantum reinforcement learning. arXiv:1811.08676v1 [quant-ph]

show abstract

Section: Explorationsupporting

confidence: 64%

Section: Explorationmentioning

confidence: 99%

Advances in quantum reinforcement learning

Dunjko

Taylor

Briegel

2017

2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…If we additionally label each percept (think positions, or directions of optimal moves in maze problems), then many well-studied reinforcement learning models (e.g. Q-Learning [14,26], Policy iteration [34] or the more recent Projective Simulation [35,36] model), together with the maze environment (with a unique winning path) do form luck-favoring pairs for all histories, so Theorem 1 applies. To further explain why this is the case (but without going into the details of these learning models), recall that in this single-win, bounded maximal time (M ) case, there is only one M -length history which has a reward.…”

Section: A Q a Q Ementioning

confidence: 99%

Quantum-Enhanced Machine Learning

2016

Self Cite

View full text Add to dashboard Cite

The emerging field of quantum machine learning has the potential to substantially aid in the problems and scope of artificial intelligence. This is only enhanced by recent successes in the field of classical machine learning. In this work we propose an approach for the systematic treatment of machine learning, from the perspective of quantum information. Our approach is general and covers all three main branches of machine learning: supervised, unsupervised, and reinforcement learning. While quantum improvements in supervised and unsupervised learning have been reported, reinforcement learning has received much less attention. Within our approach, we tackle the problem of quantum enhancements in reinforcement learning as well, and propose a systematic scheme for providing improvements. As an example, we show that quadratic improvements in learning efficiency, and exponential improvements in performance over limited time periods, can be obtained for a broad class of learning problems.

show abstract

“…The glow parameter is relevant in environments with delayed rewards such as the GridWorld [23] discussed in section 4. For a more detailed description of PS we refer the reader to [38,44,45].…”

Section: Projective Simulationmentioning

confidence: 99%

Photonic architecture for reinforcement learning

et al. 2020

View full text Add to dashboard Cite

The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in quantum optical experiments appears to be within the reach of near-term technology.OPEN ACCESS RECEIVED promising features as compared to electronic processors [27][28][29][30]. For instance, nanosecond-scale routing and reconfigurability have already been demonstrated [31][32][33], while encoding information in photons enables decision-making at the speed of light, only limited by the generation and detection rates. Moreover, the use of phase-change materials for in-memory information processing [34] promises to enhance the energy efficiency, since their properties can be modified without continuous external intervention [35,36]. Importantly, since the architecture uses single photons, decision-making is fueled by genuine quantum randomness. This feature marks a fundamental departure from pseudorandom number generation in conventional devices. (ii) The second contribution is the development of a specific variant of PS based on binary decision trees (tree-PS, or t-PS for short), which is closely connected to the standard PS and suitable for the implementation on a photonic circuit. Furthermore, we discuss how this variant enables key features of artificial intelligence, namely abstraction and generalization [37,38].The Article is structured as follows. In section 2 we summarize the theoretical framework of RL, exemplified by three common approaches: SARSA, Q-learning, and PS. In section 3 we describe the blueprint for a fully integrated, photonic RL agent. We then numerically investigate its performance within two standard RL tasks and under realistic experimental imperfections in section 4. Finally, in section 5 we discuss promising features of this architecture within the context of t-PS.

show abstract

Meta-learning within Projective Simulation

Cited by 32 publications

References 42 publications

Advances in quantum reinforcement learning

Advances in quantum reinforcement learning

Quantum-Enhanced Machine Learning

Photonic architecture for reinforcement learning

Contact Info

Product

Resources

About