Performance boost of time-delay reservoir computing by non-resonant clock cycle

Abstract

The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the clock cycle for a general model. Typically, these two time scales are considered to be equal. Here we show that the case of equal or rationally related time-delay and clock cycle could be actively detrimental and leads to an increase of the approximation error of the reservoir. In particular, we can show that non-resonant ratios of these time scales have maximal memory capacities. We achieve this by translating the periodically driven delay-dynamical system into an equivalent network. Networks that originate from a system with resonant delay-times and clock cycles fail to utilize all of their degrees of freedom, which causes the degradation of their performance.

Limitations of the Recall Capabilities in Delay-Based Reservoir Computing Systems

“…Depicted by the vertical red dashed lines are multiples of the input period time T at which the total memory capacity MC drops again significantly to around 40. A drop in the linear memory capacity was discussed in the paper by Stelzer et al [38] and explained by the fact that resonances between the delay time τ and the input period time T concludes in a sparse connection between the virtual nodes.…”

Emulating complex networks with a single delay differential equation

“…Hence, they can be employed in Machine-Learning applications with weight training. In our recent paper [50], we show that a single DDE can emulate a deep neural network and perform various computational tasks successfully. More specifically, the work [50] derives a multilayer neural network from a delay system with modulated feedback terms.…”

“…In our recent paper [50], we show that a single DDE can emulate a deep neural network and perform various computational tasks successfully. More specifically, the work [50] derives a multilayer neural network from a delay system with modulated feedback terms. This neural network is trained by gradient descent using backpropagation and applied to machine-learning tasks.…”

“…Machine Learning is another rapidly developing application area of delay systems [29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47]. It is shown recently that DDEs can successfully realize a reservoir computing setup, theoretically [41][42][43][44]46,[48][49][50], and implemented in optoelectronic hardware [30,32,39]. In time-delay reservoir computing, a single DDE with either one or a few variables is used for building a ring network of coupled maps with fixed internal weights and fixed input weights.…”

“…In Sect. 4, we review a special but practically important case of delay systems with a linear instantaneous part and nonlinear delayed feedback containing an affine combination of the delayed variables; originally, these results have been derived in [50].…”