Translation invariance in a network of oscillatory units

Abstract

One of the important features of the human visual system is that it is able to recognize objects in a scale and translational invariant manner. However, achieving this desirable behavior through biologically realistic networks is a challenge.The synchronization of neuronal firing patterns has been suggested as a possible solution to the binding problem (where a biological mechanism is sought to explain how features that represent an object can be scattered across a network, and yet be unified). This observation has led to neurons being modeled as oscillatory dynamical units. It is possible for a network of these dynamical units to exhibit synchronized oscillations under the right conditions. These network models have been applied to solve signal deconvolution or blind source separation problems. However, the use of the same network to achieve properties that the visual sytem exhibits, such as scale and translational invariance have not been fully explored.Some approaches investigated in the literature (Wallis, 1996) involve the use of non-oscillatory elements that are arranged in a hierarchy of layers. The objects presented are allowed to move, and the network utilizes a trace learning rule, where a time averaged output value is used to perform Hebbian learning with respect to the input value. This is a modification of the standard Hebbian learning rule, which typically uses instantaneous values of the input and output.In this paper we present a network of oscillatory amplitude-phase units connected in two layers. The types of connections include feedforward, feedback and lateral. The network consists of amplitude-phase units that can exhibit synchronized oscillations. We have previously shown that such a network can segment the components of each input object that most contribute to its classification. Learning is unsupervised and based on a Hebbian update, and the architecture is very simple.We extend the ability of this network to address the problem of translational invariance. We show that by adopting a specific treatment of the phase values of the output layer, the network exhibits translational invariant object representation. The scheme used in training is as follows. The network is presented with an input, which then moves. During the motion the amplitude and phase of the upper layer units is not reset, but continues with the past value before the introduction of the object in the new position. Only the input layer is changed instantaneously to reflect the moving object. The network behavior is such that it categorizes the translated objects with the same label as the stationary object, thus establishing an invariant categorization with respect to translation. This is a promising result as it uses the same framework of oscillatory units that achieves synchrony, and introduces motion to achieve translational invariance.

Unsupervised Segmentation With Dynamical Units

“…We are indeed working towards an integrated model to account for both segmentation and inference in the presence of partial information [30]. We have also shown the ability of the network presented in this paper to achieve translation invariant encoding of objects, with a minor modification to the learning rule [27].…”

“…The Hebbian learning rule in (12) is modified to use as follows: (14) The effect of the trace learning rule is to establish equivalence between translated versions of an object. In [27], we show that it is possible to obtain translation invariance of pixels for 8 8 objects (i.e., for displacements of 37.5% of the image size) as shown in Fig. 2.…”

“…There are several fruitful extensions of our model, which we have begun pursuing [27], [30]. Further work needs to be done to investigate the robustness of oscillatory networks in the handling of geometric distortions.…”

“…This allows the categorization of the object at one location to be related to translated versions of the same object. We have been able to demonstrate limited translational variance [27] with such an approach. The main idea is to use a trace learning rule, as proposed by Wallis [28], which allows associations to be formed between translated versions of a given object.…”

An optimization approach to achieve unsupervised segmentation and binding in a dynamical network

“…We are indeed working towards an integrated model to account for both segmentation and inference in the presence of partial information [17]. We have also shown the ability of the network presented in this paper to achieve translation invariant encoding of objects, with a minor modification to the learning rule [18].…”