Parallel photonic accelerator for decision making using optical spatiotemporal chaos

Abstract

Photonic accelerators have attracted increasing attention for use in artificial intelligence applications. The multi-armed bandit problem is a fundamental problem of decision making using reinforcement learning. However, to the best of our knowledge, the scalability of photonic decision making has not yet been demonstrated in experiments because of the technical difficulties in the physical realization. We propose a parallel photonic decision-making system to solve large-scale multi-armed bandit problems using optical spatiotemporal chaos. We solved a 512-armed bandit problem online, which is larger than those in previous experiments by two orders of magnitude. The scaling property for correct decision making is examined as a function of the number of slot machines, evaluated as an exponent of 0.86. This exponent is smaller than that in previous studies, indicating the superiority of the proposed parallel principle. This experimental demonstration facilitates photonic decision making to solve large-scale multi-armed bandit problems for future photonic accelerators.

Harnessing microcomb-based parallel chaos for random number generation and optical decision making

“…Optical chaos, with its fast and complex internal time evolution, is a powerful entropy source that can be used for exploration purposes 45 . Chaotic lasers have been successfully employed to solve the multi-armed bandit problem (MAB) 29 , 46 , 47 , a fundamental problem of reinforcement learning. To scale up the problem exponentially, a parallel scheme is required that employs parallel chaotic signals or is based on time-division-multiplexing 48 .…”

Photonic decision making using optical spatiotemporal chaos generated from spatial light modulator

“…We experimentally perform photonic decision making for solving the multi-armed bandit problem when the number of slot machine N is changed. Figure 2(a) shows the correct decision rate (CDR) as the number of plays is changed for different number of slot machines N [7]. The CDR is defined as the ratio of selecting the slot machine with the maximum hit probability to the total number of selection.…”

“…Figure 1(a) shows our experimental setup for photonic decision making using optical spatiotemporal chaos [7]. A semiconductor laser is used as a light source.…”

“…Experimental setup for photonic decision making using optical spatiotemporal chaos generated from a semiconductor laser, spatial light modulator, and camera. L, lens; MO, microscope objective; NDF, neutral-density filter; PC, personal computer; PL, polarizer; SF, spatial filter; SLM, spatial light modulator [7].…”

“…Figure 2. (a) Correct decision rate (CDR) as a function of the number of plays for different number of slot machines N. (b)Scaling characteristics of the number of plays for CDR of 0.95 as a function of the number of slot machines N. Red: proposed method using SLM experiment, black: laser-chaos-based method, blue: Thompson sampling, and green: UCB1tuned method[7].…”