Rapid time series prediction with a hardware-based reservoir computer

Abstract

Reservoir computing is a neural network approach for processing time-dependent signals that has seen rapid development in recent years. Physical implementations of the technique using optical reservoirs have demonstrated remarkable accuracy and processing speed at benchmark tasks. However, these approaches require an electronic output layer to maintain high performance, which limits their use in tasks such as time-series prediction, where the output is fed back into the reservoir. We present here a reservoir computing scheme that has rapid processing speed both by the reservoir and the output layer. The reservoir is realized by an autonomous, time-delay, Boolean network configured on a field-programmable gate array. We investigate the dynamical properties of the network and observe the fading memory property that is critical for successful reservoir computing. We demonstrate the utility of the technique by training a reservoir to learn the short-and long-term behavior of a chaotic system. We find accuracy comparable to state-of-the-art software approaches of similar network size, but with a superior real-time prediction rate up to 160 MHz.Reservoir computers are well-suited for machine learning tasks that involve processing timevarying signals such as those generated by human speech, communication systems, chaotic systems, weather systems, and autonomous vehicles. Compared to other neural network techniques, reservoir computers can be trained using less data and in much less time. They also possess a large network component, called the reservoir, that can be re-used for different tasks. These advantages have motivated searches for physical implementations of reservoir computers that achieve high-speed and real-time information processing, including opto-electronic and electronic devices. Here, we develop an electronic approach using an autonomous, time-delay, Boolean network configured on a field-programmable gate array (FPGA). These devices allow for complex networks consisting of 1,000's of nodes with arbitrary network topology. Time-delays can be incorporated along network links, thereby allowing for extremely high-dimension reservoirs. The characteristic time scale of a network node is less than a nanosecond, allowing for information processing in the GHz regime. Further, because the reservoir state is Boolean rather than realvalued, calculation of an output from the reservoir state can be done rapidly with synchronous FPGA logic. We use such a reservoir computer for the challenging task of forecasting the dynamics of a chaotic system. This work paves the way for low-cost, compact reservoir computers that can be embedded in various commercial and industrial systems for real-time information processing.

Forecasting chaotic systems with very low connectivity reservoir computers

“…(1), then perform prediction of the Lorenz system after that segment with Eq. (5). Because the Lorenz system is chaotic, forecasting must eventually fail.…”

Constructive Role of Noise for High-Quality Replication of Chaotic Attractor Dynamics Using a Hardware-Based Reservoir Computer

“…A qualitative comparison between the original chaotic dynamics and its experimentally generated replica was done previously [8,9]. Here we go one step further and quantify the performance of our system for the long-term reconstruction of the dynamics.…”

“…Autonomous replication of chaotic systems has been demonstrated both experimentally [8,9] and by numerical simulations [10]. However, it has been observed that such autonomous reservoir computers can settle on a * miguel@ifisc.uib-csic.es dynamical behavior different from the one they have been trained for [9,10]. We go beyond previous approaches by demonstrating that noise added to the input data can result in better overall replication of the desired chaotic dynamics and enhance robustness against parameter variations.…”

“…Here we explore the noise-induced performance enhancement in autonomous attractor replication of a chaotic dynamical system using a hardware-implemented reservoir-computing (RC) system. Autonomous replication of chaotic systems has been demonstrated both experimentally [8,9] and by numerical simulations [10]. However, it has been observed that such autonomous reservoir computers can settle on a * miguel@ifisc.uib-csic.es dynamical behavior different from the one they have been trained for [9,10].…”

Chaotic time series prediction by noisy echo state network

“…Reservoir computing is a new approach to constructing recurrent neural networks. In particular, the echo state network (ESN) is a practical method for implementing a reservoir computer [1][2][3] and has been shown to be effective, for example, for chaotic time series prediction [4][5][6][7][8][9][10]. When applied to time series prediction, the ESN usually has three layers, i.e., the input layer consisting of a single node from which input data are generated, the reservoir layer consisting of sparsely coupled multiple nodes (reservoir network), and the output layer consisting of a single node from which predicted values are provided.…”