Multiplex visibility graphs to investigate recurrent neural network dynamics

Abstract

A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.

Reservoir Computing Approaches for Representation and Classification of Multivariate Time Series

“…Each model is configured with the same hyperparameters in all the experiments. Since reservoirs are sensitive to the hyperparameter setting [47], a fine-tuning with independent cross-validation for each task is usually more important in classic RC models than in RNNs trained with gradient descent, such as LSTM and GRU. Nevertheless, we show that the proposed rmESN achieves competitive results even with fixed the hyperparameters.…”

Time series analysis via network science: Concepts and algorithms

“…Bianchi et al (2017) used multiplex visibility networks method based on the weighted HVG algorithm (Section 3.1.5) where the edge weights are given by: , incorporating temporal and amplitude information of the data. They studied topological measures of the networks to characterize neuron activations.…”

Visibility Graph Analysis of Intraspinal Pressure Signal Predicts Functional Outcome in Spinal Cord Injured Patients

“…if node i corresponds to a future time than node j, and are visible to each other, then there is connection from i®j but not i¬j 35 . There is also a technique to map multivariate time series into multiplex graphs 36 that may enable analysis of multi-modality monitoring data from the injury site e.g. ISP, tissue oxygen and microdialysis.…”