Spatial vs temporal continuity in view invariant visual object recognition learning

Abstract

We show in a 4-layer competitive neuronal network that continuous transformation learning, which uses spatial correlations and a purely associative (Hebbian) synaptic modification rule, can build view invariant representations of complex 3D objects. This occurs even when views of the different objects are interleaved, a condition where temporal trace learning fails. Human psychophysical experiments showed that view invariant object learning can occur when spatial but not temporal continuity applies because of interleaving of stimuli, although sequential presentation, which produces temporal continuity, can facilitate learning. Thus continuous transformation learning is an important principle that may contribute to view invariant object recognition.

The role of sequence order in determining view canonicality for novel wire-frame objects

“…Viewing sequence thus matters but most likely matters a good deal less than does 3-D form. This conclusion is in line with previous results suggesting that pure spatial continuity between images is enough to induce the kind of image binding described in temporal association experiments (Perry, Rolls, & Stringer, 2006) and also with the finding that previous experience with specific object rotations is not necessary for robust extrapolation across views (Wang, Obama, Yamashita, Sugihara, & Tanaka, 2005). In general, it may be the case that the sequence of views seen by the observer makes a small enough contribution to the processes governing canonicality, recognition, or generalization across views that removing it from the equation does not cause the visual system to break in a profound way .…”

Invariant object recognition with trace learning and multiple stimuli present during training

“…In most previous studies of invariance learning in hierarchical networks that model the ventral visual stream, only one stimulus is presented at a time during training (Wallis and Rolls 1997;Rolls and Milward 2000;Elliffe et al 2002;Perry et al 2006;Rolls and Stringer 2006;Stringer et al 2006). However, an important problem in understanding natural vision is how the brain can build invariant representations of individual objects even when multiple objects are present in a scene.…”

Finding and recognizing objects in natural scenes: complementary computations in the dorsal and ventral visual systems

“…The finding that temporal contiguity in the absence of reward is sufficient to lead to view invariant object representations in the inferior temporal visual cortex has been confirmed (Li and DiCarlo, 2008, 2010, 2012). The importance of temporal continuity in learning invariant representations has also been demonstrated in human psychophysics experiments (Perry et al, 2006; Wallis, 2013). Some other simulation models are also adopting the use of temporal continuity as a guiding principle for developing invariant representations by learning (Wiskott and Sejnowski, 2002; Wiskott, 2003; Wyss et al, 2006; Franzius et al, 2007), and the temporal trace learning principle has also been applied recently (Isik et al, 2012) to HMAX (Riesenhuber and Poggio, 2000; Serre et al, 2007c).…”

“…(An interesting example is that representations of individual people or objects invariant with respect to pose (e.g., standing, sitting, walking) can be learned by VisNet, or representations of pose invariant with respect to the individual person or object can be learned by VisNet depending on the order in which the identical images are presented during training Webb and Rolls, 2014.) Indeed, we developed these hypotheses (Rolls, 1992, 1995, 2012; Wallis et al, 1993) into a model of the ventral visual system that can account for translation, size, view, lighting, and rotation invariance (Wallis and Rolls, 1997; Rolls and Milward, 2000; Stringer and Rolls, 2000, 2002, 2008; Rolls and Stringer, 2001, 2006, 2007; Elliffe et al, 2002; Perry et al, 2006, 2010; Stringer et al, 2006, 2007; Rolls, 2008, 2012). Consistent with the hypothesis, we have demonstrated these types of invariance (and spatial frequency invariance) in the responses of neurons in the macaque inferior temporal visual cortex (Rolls et al, 1985, 1987, 2003; Rolls and Baylis, 1986; Hasselmo et al, 1989; Tovee et al, 1994; Booth and Rolls, 1998).…”