Reservoir computing approaches for representation and classification of multivariate time series

Bianchi, Filippo Maria; Scardapane, Simone; Løkse, Sigurd; Jenssen, Robert

doi:10.48550/arxiv.1803.07870

Cited by 6 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The desired -task dependent -output is then generated by a readout layer (usually linear) trained to match the states with the desired outputs. Despite the simplified training protocol, ESNs are universal function approximators 5 and have shown to be effective in many relevant tasks [6][7][8][9][10][11][12] .…”

Section: Introductionmentioning

confidence: 99%

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Verzelli

Alippi

Livi

2019

Sci Rep

View full text Add to dashboard Cite

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behavior. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of criticality. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory–nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behavior in phase space characterized by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking.

show abstract

Section: Introductionmentioning

confidence: 99%

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Verzelli

Alippi

Livi

2019

Sci Rep

View full text Add to dashboard Cite

show abstract

“…To ease readability, we opted to present just a natural language description of each query, instead of its formal representation in LASER. 6 The experiments were run on a computer with an Intel i7 8th gen hexacore processor, with GPU Nvidia 1050 Ti, and 8GB of RAM. We note that pre-processing the input data (for LASER) as well as encoding the input for the networks took only a few milliseconds per time step, which is why it is not reported individually, and that training the networks required on average not more than ten minutes for both kinds of networks, which is only once before applying the network, and, once trained, neural networks never required more than 300 µs to produce the output for one time step in any of the experiments.…”

Section: Description Of Experiments and Resultsmentioning

confidence: 99%

“…In this paper, we investigate the feasibility of using Deep Neural Networks to approximate stream reasoning with LASER, to take advantage of their high processing speed. We explore two types of neural networks, namely Convolutional Neural Networks (CNN) [28] and Recurrent Neural Networks (RNNs) [25], which have been shown to obtain good results when applied to time-annotated data problems, such as time series forecasting and classification [6,12]. For our experiments, we consider a real dataset with time-annotated sensor data on traffic, pollution and weather conditions, and explore different types of LASER queries, in order to cover different expressive features of its language.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Faster than LASER -- Towards Stream Reasoning with Deep Neural Networks

Ferreira¹,

Diogo²,

Jardim-Goncalves³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the constant increase of available data in various domains, such as the Internet of Things, Social Networks or Smart Cities, it has become fundamental that agents are able to process and reason with such data in real time. Whereas reasoning over time-annotated data with background knowledge may be challenging, due to the volume and velocity in which such data is being produced, such complex reasoning is necessary in scenarios where agents need to discover potential problems and this cannot be done with simple stream processing techniques. Stream Reasoners aim at bridging this gap between reasoning and stream processing and LASER is such a stream reasoner designed to analyse and perform complex reasoning over streams of data. It is based on LARS, a rule-based logical language extending Answer Set Programming, and it has shown better runtime results than other state-of-the-art stream reasoning systems. Nevertheless, for high levels of data throughput even LASER may be unable to compute answers in a timely fashion. In this paper, we study whether Convolutional and Recurrent Neural Networks, which have shown to be particularly well-suited for time series forecasting and classification, can be trained to approximate reasoning with LASER, so that agents can benefit from their high processing speed.

show abstract

“…Reservoir Computing (RC) is a less conventional method for using Recurrent Neural Networks that has been widely used in applications such as time-series forecasting (Deihimi & Showkati, 2012;Bianchi et al, 2015b;a), process modelling (Rodan et al, 2017), speech analysis (Trentin et al, 2015), and classification of multivariant time series (Bianchi et al, 2018). RC models conceptually divide time-series processing into two components: (i) representation of temporal structure in the input stream through a non-adaptable dynamic reservoir (generated through the feedback-driven dynamics of a randomly drawn RNN), and (ii) an easy-toadapt readout from the reservoir.…”

Section: Reservoir Computingmentioning

confidence: 99%

Randomly Weighted, Untrained Neural Tensor Networks Achieve Greater Relational Expressiveness

Hong¹,

Pavlic²

2020

Preprint

View full text Add to dashboard Cite

Neural Tensor Networks (NTNs), which are structured to encode the degree of relationship among pairs of entities, are used in Logic Tensor Networks (LTNs) to facilitate Statistical Relational Learning (SRL) in first-order logic. In this paper, we propose Randomly Weighted Tensor Networks (RWTNs), which incorporate randomly drawn, untrained tensors into an NTN encoder network with a trained decoder network. We show that RWTNs meet or surpass the performance of traditionally trained LTNs for Semantic Image Interpretation (SII) tasks that have been used as a representative example of how LTNs utilize reasoning over first-order logic to exceed the performance of solely data-driven methods. We demonstrate that RWTNs outperform LTNs for the detection of the relevant part-of relations between objects, and we show that RWTNs can achieve similar performance as LTNs for object classification while using fewer parameters for learning.

show abstract

Reservoir computing approaches for representation and classification of multivariate time series

Cited by 6 publications

References 0 publications

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Faster than LASER -- Towards Stream Reasoning with Deep Neural Networks

Randomly Weighted, Untrained Neural Tensor Networks Achieve Greater Relational Expressiveness

Contact Info

Product

Resources

About