Reducing reservoir computer hyperparameter dependence by external timescale tailoring

Abstract

Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced if timescales of the reservoir are tailored to the specific task. Our results are mainly relevant for temporal tasks requiring memory of past inputs, for example chaotic timeseries prediciton. We consider various methods of including task specific timescales in the reservoir computing approach and demonstrate the universality of our message by looking at both time-multiplexed and spatially multiplexed reservoir computing.

Exploring non-steady-state charge transport dynamics in information processing: insights from reservoir computing

“…In the initial study of RC based on nonlinear dynamic node by Appeltant et al, it has been proposed that proper selection of the characteristic time scale of the nonlinear node with respect to the separation time θ virtual nodes is crucial for the performance of the RC. Similar ideas were later followed by many researches [62,63], based on both delay dynamical systems and non-steady state dynamic systems. In this study, we demonstrate quantitatively how the characteristic time of the non-steady-state proton-coupled charge transport system influence of the RC performance.…”

“…The pursuit of optimal parameters remains a pivotal challenge for specific systems, notably in the realm of hardware implemented reservoir. There has also been considerable in-depth advancement in optimizing the architecture of RC, enhancing its efficacy [61,62].…”

Reservoir computing using spin-VCSELs - the influence of timescales and data injection schemes

“…Lastly, it is noted that the choice of the delay time τ changes the bifurcation structure (it determines the distance of the saw tooth like looking tongues [59]) and larger τ increases the level of multistability [56,72,73]). Additionally, the ratio between τ and T changes the performance due to resonance effects [23,24,74] and due to the influence on the memory of the reservoir [49,74]. We always use an irrational ratio of (2) between both.…”

“…For a one step ahead x-to-x prediction task the vector x(k) is the input and o(k) = x(k+1) is the target. It is noted that the sampling step has an impact both on the difficulty of the task and on its memory requirements [49,82]. Our focus is on the interrelation between the bifurcation structure and the performance, which is why only one task is used for the discussion (other task have been performed to ensure that the results are robust and are shown in the Supplementary material).…”

“…Besides the bifurcation structure, there are other ways to improve the performance of a RC setup, e.g. by means of state matrix extensions [27,[46][47][48][49], input delay [49][50][51], additional nonlinear regression at the output [52,53], or the addition of physics informed information [54,55]. These methods are not the focus of the present study, but are complimentary to the work we present.…”