Attractive Periodic Sets in Discrete-Time Recurrent Networks (with Emphasis on Fixed-Point Stability and Bifurcations in Two-Neuron Networks)

Abstract

We perform a detailed xed-point analysis of two-unit recurrent neural networks with sigmoid-shaped transfer functions. Using geometrical arguments in the space of transfer function derivatives, we partition the network state-space into distinct regions corresponding to stability types of the xed points. Unlike in the previous studies, we do not assume any special form of connectivity pattern between the neurons, and all free parameters are allowed to vary. We also prove that when both neurons have excitatory self-connections and the mutual interaction pattern is the same (i.e., the neurons mutually inhibit or excite themselves), new attractive xed points are created through the saddle-node bifurcation. Finally, for an N-neuron recurrent network, we give lower bounds on the rate of convergence of attractive periodic points toward the saturation values of neuron activations, as the absolute values of connection weights grow.

Reservoir Computing with Output Feedback

“…a single globally and asymptotically stable fixed-point exists, or bi-stable, i.e. two fixed-points are separated by a saddle [137,138,139].…”

Stability of Quasi-Periodic Orbits in Recurrent Neural Networks

“…Lim k→∞ a(ϕ) + a(P(ϕ)) + · · · + a(P k−1 (ϕ)) k (20) where a(ϕ) = P(ϕ) − ϕ P(ϕ) = ϕ + 2πυ(mod 2π), and a(ϕ) is called the angular function [11]. This number is estimated considering the rotation around the origin…”

“…Passeman [16] obtains some experimental results such as the coexistence of periodic cycles, quasi-periodic trajectories and chaotic attractors. Tino gives in [20] the position, number and stability types of fixed points of a two-neuron discrete recurrent network with nonzero weights.…”

Watching The Transients: Viewing a Simple Recurrent Network As a Limited Counter

“…Since a bifurcation for a discrete-time dynamical system occurs at a fixed point when the dominant eigenvalue changes from < 1 to > 1 (or from > -1 to < -1) (e.g. see Tino, et al, 1995b), the need for divergence creates a potential bifurcation.…”