Indirect training of a spiking neural network for flight control via spike-timing-dependent synaptic plasticity

Abstract

Recently, spiking neural networks (SNNs) have been shown capable of approximating the dynamics of biological neuronal networks, and of being trainable by biologicallyplausible learning mechanisms, such as spike-timing-dependent synaptic plasticity. Numerical simulations also support the possibility that they may possess universal function approximation abilities. However the effectiveness of training algorithms to date is far inferior to those of other artificial neural networks. Moreover, they rely on directly manipulating the SNN weights, which may not be feasible in a number of their potential applications. This paper presents a novel indirect training approach to modulate spike-timing-dependent plasticity (STDP) in an action SNN that serves as a flight controller without directly manipulating its weights. A critic SNN is directly trained with a reward-based Hebbian approach to send spike trains to the action SNN, which in turn controls the aircraft and learns via STDP. The approach is demonstrated by training the action SNN to act as a flight controller for stability augmentation. Its performance and dynamics are analyzed before and after training through numerical simulations and Poincaré maps.

A Radial Basis Function Spike Model for Indirect Learning via Integrate-and-Fire Sampling and Reconstruction Techniques

“…In [15], the authors proposed an indirect training method based on a Poison spike model and demonstrated it on a network of Izhikevich neurons. In this work it was found that the adaptive critic architecture described in Section 3 could be implemented without modeling the neuron response in closed form.…”

“…In this work it was found that the adaptive critic architecture described in Section 3 could be implemented without modeling the neuron response in closed form. However, the effectiveness of the approach in [15] was limited in that, due to the use of a stochastic Poison model, the Izhikevich SNN could not converge to the optimal control law. Therefore, in this paper, a new deterministic spike model is proposed and implemented by deriving the training equations in closed form, using a leaky integrate-and-fire (LIF) SNN.…”

“…For simplicity, these effects are assumed negligible in this paper. However, the reader is referred to [15] for a technique that can be used to incorporate these effects in the above SNN model.…”

“…To accomplish (15) and 16, two error metrics defined as a function of the actual (decoded) actor and critic outputs and of the desired actor and critic outputs (13) and 14must be minimized by the chosen training algorithm, as explained in the following subsection.…”

“…While available in closed analytic form, these signals typically are represented by piece-wise continuous, multi-valued (or many-to-one), and nondifferentiable functions that are difficult to adapt or update using optimization or reinforcement-learning algorithms. Furthermore, stimulation patterns typically are generated by spike models that are stochastic, such as the Poisson spike model [5,[13][14][15]. Thus, even when the spike model is optimized, it does not allow for precise timing of pre-and post-synaptic firings, and as a result, may induce undesirable changes in the synaptic weights.…”