Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

Noise and time delay are two elements that are associated with many natural systems, and often they are sources of complex behaviors. Understanding of this complexity is yet to be explored, particularly when both elements are present. As a step to gain insight into such complexity for a system with both noise and delay, we investigate such delayed stochastic systems both in dynamical and probabilistic perspectives. A Langevin equation with delay and a random-walk model whose transition probability depends on a fixed time-interval past (delayed random walk model) are the subjects of in depth focus. As well as considering relations between these two types of models, we derive an approximate Fokker-Planck equation for delayed stochastic systems and compare its solution with numerical results.

Abstract.A technique is given for detecting deterministic dynamics in time series. Some stochastic difference equations, called KM^O-Langevin equations, are extracted directly from given data. We can find deterministic dynamics in time series by evaluating the magnitude of innovation part of the KM2O-Langevin equations. We can further find chaotic dynamics in time series by predicting it from the viewpoint of the theory of KlVhO-Langevin equations.We apply our method to the data of measles and chicken pox, which are also treated by G.Sugihara and R.M.May in [1]. The result of numerical experiments indicates that there seem to exist some deterministic dynamics in both time series. It also suggests, however, that the data of measles seems to be chaotic while that of chicken pox not, which corresponds to the result of G.Sugihara and R.M.May. §1. IntroductionThere are a lot of systems in the world, whose behavior as a whole is never understandable if we only view their components separately. Such systems, called complex systems, arise from a variety of origins of complexity such as stochastic structure, deterministic chaos and so on. This feature of complex systems has the result that a priori parametric statistical models (e.g. ARMA model, linear regression model) may fail to catch the underlying structure arising from the complex systems which lies behind the data.Therefore, we must check the validity of the preconditions that is assumed before data analysis. We call such an approach toward data analysis a qualitative approach in contrast to quantitative approaches such as parametric statistical models. One of the authors has presented a precondition-

The development of hardware neural networks, including neuromorphic hardware, has been accelerated over the past few years. However, it is challenging to operate very large-scale neural networks with low-power hardware devices, partly due to signal transmissions through a massive number of interconnections. Our aim is to deal with the issue of communication cost from an algorithmic viewpoint and study learning algorithms for energy-efficient information processing. Here, we consider two approaches to finding spatially arranged sparse recurrent neural networks with the high cost-performance ratio for associative memory. In the first approach following classical methods, we focus on sparse modular network structures inspired by biological brain networks and examine their storage capacity under an iterative learning rule. We show that incorporating long-range intermodule connections into purely modular networks can enhance the cost-performance ratio. In the second approach, we formulate for the first time an optimization problem where the network sparsity is maximized under the constraints imposed by a pattern embedding condition. We show that there is a tradeoff between the interconnection cost and the computational performance in the optimized networks. We demonstrate that the optimized networks can achieve a better cost-performance ratio compared with those considered in the first approach. We show the effectiveness of the optimization approach mainly using binary patterns and apply it also to grayscale image restoration. Our results suggest that the presented approaches are useful in seeking more sparse and less costly connectivity of neural networks for the enhancement of energy efficiency in hardware neural networks.

No abstract