Optimizing memory in reservoir computers

Abstract

A reservoir computer is a way of using a high dimensional dynamical system for computation. One way to construct a reservoir computer is by connecting a set of nonlinear nodes into a network. Because the network creates feedback between nodes, the reservoir computer has memory. If the reservoir computer is to respond to an input signal in a consistent way (a necessary condition for computation), the memory must be fading; that is, the influence of the initial conditions fades over time. How long this memory lasts is important for determining how well the reservoir computer can solve a particular problem. In this paper, I describe ways to vary the length of the fading memory in reservoir computers. Tuning the memory can be important to achieve optimal results in some problems; too much or too little memory degrades the accuracy of the computation.

Universal scaling between wave speed and size enables nanoscale high-performance reservoir computing based on propagating spin-waves

“…We will discuss the details of the two methods. Our results are consistent with previous studies; the requirement of memory was discussed in (54), and the second-order nonlinear terms with a time delay in (55).…”

“…Here, M † is the pseudoinverse of the matrix M. This is nothing but a least-square estimation for the cost function of l.h.s minus r.h.s of (54). We may include the Tikhonov regularization term.…”

Time Shifts to Reduce the Size of Reservoir Computers

“…Using more nodes allows the covariance rank to be larger, since it is limited by the size of the reservoir. For a particular rank there is a spread in testing error because there are other factors that affect the error, such as the reservoir Lyapunov exponents [33] or the memory [34], but the main effect of the number of nodes is to increase the possible rank. The point of Fig.…”

“…It is generally accepted that the larger memory capacity for a reservoir computer leads to lower testing errors, although [34] pointed out that the memory capacity must be matched to the problem at hand, so memory capacity in some cases can be too large. We may ask if larger reservoir computers increase the memory capacity; Fig.…”

“…Memory capacity can be affected by the number of nodes, although our data suggests that covariance rank has a larger effect. It has been demonstrated that too much memory capacity can degrade the performance of a reservoir computer [33,34].…”