Decision making for the multi-armed bandit problem using lag synchronization of chaos in mutually coupled semiconductor lasers

Mihana, Takatomo; Mitsui, Yusuke; Takabayashi, Mizuho; Kanno, Kazutaka; Sunada, Satoshi; Naruse, Makoto; Uchida, Atsushi

doi:10.1364/oe.27.026989

Cited by 29 publications

(13 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where Δ and Ω represent the step sizes of the threshold adjuster when the results of the selected slot machine are "hit" and "miss," respectively. We use a new definition of Δ and Ω [18] to avoid excessively large or small values (Δ = 1 and Ω = ( P1 + P2 )/(2 − ( P1 + P2 )) were used in [15,16]). Pi is the estimated hit probability for slot machine i (S i ) and is obtained as follows:…”

Section: Tug-of-war Methodsmentioning

confidence: 99%

“…The tug-of-war method provides faster decision making than software algorithms [7]. Recently, photonic implementations of the tug-of-war method have been demonstrated using quantum dots [11,12], single photons [13], entangled photons [14], and chaotic semiconductor lasers [15][16][17][18][19][20][21][22][23]. Furthermore, photonic decision making of the multi-armed bandit problem has been reported using chaotic temporal waveforms with a threshold [15][16][17], lag synchronization of chaos in coupled semiconductor lasers [18], laser network dynamics [19,20], and mode-switching dynamics in a ring-cavity laser [21].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive decision making using a chaotic semiconductor laser for multi-armed bandit problem with time-varying hit probabilities

Oda

Mihana

Kanno

et al. 2022

NOLTA

Self Cite

View full text Add to dashboard Cite

We numerically demonstrate the principle of adaptive decision making for solving multi-armed bandit problems in dynamically changing reward environments. We use the tugof-war method by comparing a threshold and a chaotic temporal waveform generated from a semiconductor laser observed in an experiment. We propose a method for detecting dynamic changes in hit probabilities by evaluating short-term standard deviations of the estimated hit probabilities. Furthermore, the threshold is forced to be initialized when changes in the hit probabilities are detected. We perform adaptive decision making in time-varying hit probabilities, including cases in which the differences in the hit probabilities are small. The proposed method paves the way for ultrafast photonic decision making in dynamically changing environments for various applications, such as cognitive wireless communications and robot control using reinforcement learning.

show abstract

Section: Tug-of-war Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptive decision making using a chaotic semiconductor laser for multi-armed bandit problem with time-varying hit probabilities

Oda

Mihana

Kanno

et al. 2022

NOLTA

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, mode competition dynamics in a ring–cavity semiconductor laser on a chip has been utilized to solve two-armed bandit problems (i.e., problems with two slot machines) 13 . The lag synchronization of chaos in mutually coupled semiconductor lasers has been used for decision making 14 , and this approach has been extended to laser networks with a large number of slot machines 15 , 16 . Furthermore, single 17 , 18 and entangled 19 – 21 photons have been utilized for photonic decision making.…”

Section: Introductionmentioning

confidence: 99%

Decision making for large-scale multi-armed bandit problems using bias control of chaotic temporal waveforms in semiconductor lasers

Morijiri

Mihana

Kanno

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

Decision making using photonic technologies has been intensively researched for solving the multi-armed bandit problem, which is fundamental to reinforcement learning. However, these technologies are yet to be extended to large-scale multi-armed bandit problems. In this study, we conduct a numerical investigation of decision making to solve large-scale multi-armed bandit problems by controlling the biases of chaotic temporal waveforms generated in semiconductor lasers with optical feedback. We generate chaotic temporal waveforms using the semiconductor lasers, and each waveform is assigned to a slot machine (or choice) in the multi-armed bandit problem. The biases in the amplitudes of the chaotic waveforms are adjusted based on rewards using the tug-of-war method. Subsequently, the slot machine that yields the maximum-amplitude chaotic temporal waveform with bias is selected. The scaling properties of the correct decision-making process are examined by increasing the number of slot machines to 1024, and the scaling exponent of the power-law distribution is 0.97. We demonstrate that the proposed method outperforms existing software algorithms in terms of the scaling exponent. This result paves the way for photonic decision making in large-scale multi-armed bandit problems using photonic accelerators.

show abstract

“…By simply adjusting the operating conditions of the two lasers, including bias current, coupling strength, and detuning frequency, various dynamical behaviors can be induced, such as mutual injection locking, period-one (P1) dynamics, period-two (P2) dynamics, quasi-periodic dynamics, and chaos. The unique temporal and spectral features found in these dynamical behaviors have been proposed, respectively, to improve performance characteristics of existing technologies, such as enhancing the bandwidth of direct modulation [1][2][3][4][5] and suppressing nonlinear distortion due to direct modulation [6][7][8], or to provide alternatives for novel applications, such as tunable microwave generation [9][10][11][12], chaotic synchronization [13][14][15][16], reservoir computing [17][18][19], and decision making [20]. For these technological applications, the bias currents of the two lasers are, in general, adjusted independently and differently so that specific characteristics or functionalities are achieved.…”

Section: Introductionmentioning

confidence: 99%

Effects of Asymmetric Coupling Strength on Nonlinear Dynamics of Two Mutually Long-Delay-Coupled Semiconductor Lasers

et al. 2022

View full text Add to dashboard Cite

This study investigates the effects of asymmetric coupling strength on nonlinear dynamics of two mutually long-delay-coupled semiconductor lasers through both experimental and numerical efforts. Dynamical maps and spectral features of dynamical states are analyzed as a function of the coupling strength and detuning frequency for a fixed coupling delay time. Symmetry in the coupling strength of the two lasers, in general, symmetrizes their dynamical behaviors and the corresponding spectral features. Slight to moderate asymmetry in the coupling strength moderately changes their dynamical behaviors from the ones when the coupling strength is symmetric, but does not break the symmetry of their dynamical behaviors and the corresponding spectral features. High asymmetry in the coupling strength not only strongly changes their dynamical behaviors from the ones when the coupling strength is symmetric, but also breaks the symmetry of their dynamical behaviors and the corresponding spectral features. Evolution of the dynamical behaviors from symmetry to asymmetry between the two lasers is identified. Experimental observations and numerical predictions agree not only qualitatively to a high extent but also quantitatively to a moderate extent.

show abstract

Decision making for the multi-armed bandit problem using lag synchronization of chaos in mutually coupled semiconductor lasers

Cited by 29 publications

References 32 publications

Adaptive decision making using a chaotic semiconductor laser for multi-armed bandit problem with time-varying hit probabilities

Adaptive decision making using a chaotic semiconductor laser for multi-armed bandit problem with time-varying hit probabilities

Decision making for large-scale multi-armed bandit problems using bias control of chaotic temporal waveforms in semiconductor lasers

Effects of Asymmetric Coupling Strength on Nonlinear Dynamics of Two Mutually Long-Delay-Coupled Semiconductor Lasers

Contact Info

Product

Resources

About