Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input

Abstract

Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly. In this work, we introduce a discrete-time version of EP with simplified equations and with reduced simulation time, bringing EP closer to practical machine learning tasks. We first prove theoretically, as well as numerically that the neural and weight updates of EP, computed by forward-time dynamics, are step-by-step equal to the ones obtained by BPTT, with gradients computed backward in time. The equality is strict when the transition function of the dynamics derives from a primitive function and the steady state is maintained long enough. We then show for more standard discrete-time neural network dynamics that the same property is approximately respected and we subsequently demonstrate training with EP with equivalent performance to BPTT. In particular, we define the first convolutional architecture trained with EP achieving ∼ 1% test error on MNIST, which is the lowest error reported with EP. These results can guide the development of deep neural networks trained with EP.Preprint. Under review.

Hierarchical Associative Memory

“…The convolutional setting was also discussed in [25]. The primitive function they develop, analog of the energy function (equation 18 in [25]), however, is just a quadratic form. Dynamical equations defined as derivatives of that function are linear.…”

Photonic kernel machine learning for ultrafast spectral analysis

“…Such architectures come with their own technical difficulties. Because they rely on co-located memory and processing resources, standard optimization algorithms can prove impractical, although novel optimization procedures have recently been put forward to overcome this obstacle by meeting their peculiar hardware design [77][78][79][80][81].…”

Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias

“…First, the network relaxes to a steady state, then the output layer is nudged towards a ground-truth target until a second steady state is reached. During the second phase, the perturbation at the output propagates to upstream layers, creating local error signals that match exactly those computed by Backpropagation Through Time (BPTT) [Ernoult et al, 2019]. The spatial locality of the learning rule prescribed by EP is highly attractive for designing energy-efficient "neuromorphic" hardware implementations of gradient-based learning algorithms.…”

“…However, previous works on EP [Scellier and Bengio, 2017, O'Connor et al, 2018, O'Connor et al, 2019, Ernoult et al, 2019 limited their experiments to the MNIST classification task and to shallow network architectures. Despite the theoretical guarantees of EP, the literature suggests that no implementation of EP has thus far succeeded to match the performance of standard deep learning approaches to train deep networks on challenging visual tasks.…”

“…where Φ denotes the scalar primitive of the dynamics, and which converges to a steady-state s * . These discrete-time equations, introduced in [Ernoult et al, 2019], are a simplification of the original version of EP for real-time dynamical systems [Scellier and Bengio, 2017], where the dynamics derive from an "energy function" E, which plays a similar role to the primitive function Φ. During the second ("nudged") phase of EP, the system evolves with an additional nudging term added to the dynamics, β being a small scalar factor:…”