Movement prediction from real-world images using a liquid state machine

Abstract

The prediction of time series is an important task in finance, economy, object tracking, state estimation and robotics. Prediction is in general either based on a wellknown mathematical description of the system behind the time series or learned from previously collected time series. In this work we introduce a novel approach to learn predictions of real world time series like object trajectories in robotics. In a sequence of experiments we evaluate whether a liquid state machine in combination with a supervised learning algorithm can be used to predict ball trajectories with input data coming from a video camera mounted on a robot participating in the RoboCup. The pre-processed video data is fed into a recurrent spiking neural network. Connections to some output neurons are trained by linear regression to predict the position of a ball in various time steps ahead. The main advantages of this approach are that due to the nonlinear projection of the input data to a high-dimensional space simple learning algorithms can be used, that the liquid state machine provides temporal memory capabilities and that this kind of computation appears biologically more plausible than conventional methods for prediction. Our results support the idea that learning with a liquid state machine is a generic powerful tool for prediction.

Predicting Performance using Approximate State Space Model for Liquid State Machines

“…Maass et al have argued that these non-linear operations performed by LSMs allow it to display high performance and universal computational capabilities [1]. Many applications are based on the belief that non linearity of the LSMs enable powerful data processing [2]- [4]. In [3] nonlinear computations were performed on time series data.…”

“…In [3] nonlinear computations were performed on time series data. In [4], LSM was used for movement prediction task and was shown to perform a non-linear technique of Kernal Principal Component Analysis. Speech recognition, time series prediction, and robot control are few of the many versatile applications for which LSM has demonstrated excellent performance [2], [5]- [7].…”

A novel digital neuromorphic architecture efficiently facilitating complex synaptic response functions applied to liquid state machines

“…LSMs have been successfully applied to several applications including speech recognition [10], vision [23], and cognitive neuroscience [11], [24]. Practical applications suffer from the fact that traditional LSMs take input in the form of spike trains.…”

Improving the Classification Performance of Liquid State Machines Based on the Separation Property

“…(b) The within-class scatter matrix: (2) where is the covariance matrix of the liquid states that correspond to class , and is as eq. 1.…”

“…Following their introduction [9], LSMs have been used in various pattern classification tasks, including speech recognition [14] and movement prediction [2]. The notion behind LSMs has also been extended to problem domains outside computational modeling, where researchers use physical mediums for the implementation of the liquid, such as a bucket of water [5] or real cell assemblies [3].…”