Invariant visual object recognition: biologically plausible approaches

Abstract

Key properties of inferior temporal cortex neurons are described, and then, the biological plausibility of two leading approaches to invariant visual object recognition in the ventral visual system is assessed to investigate whether they account for these properties. Experiment 1 shows that VisNet performs object classification with random exemplars comparably to HMAX, except that the final layer C neurons of HMAX have a very non-sparse representation (unlike that in the brain) that provides little information in the single-neuron responses about the object class. Experiment 2 shows that VisNet forms invariant representations when trained with different views of each object, whereas HMAX performs poorly when assessed with a biologically plausible pattern association network, as HMAX has no mechanism to learn view invariance. Experiment 3 shows that VisNet neurons do not respond to scrambled images of faces, and thus encode shape information. HMAX neurons responded with similarly high rates to the unscrambled and scrambled faces, indicating that low-level features including texture may be relevant to HMAX performance. Experiment 4 shows that VisNet can learn to recognize objects even when the view provided by the object changes catastrophically as it transforms, whereas HMAX has no learning mechanism in its S–C hierarchy that provides for view-invariant learning. This highlights some requirements for the neurobiological mechanisms of high-level vision, and how some different approaches perform, in order to help understand the fundamental underlying principles of invariant visual object recognition in the ventral visual stream.Electronic supplementary materialThe online version of this article (doi:10.1007/s00422-015-0658-2) contains supplementary material, which is available to authorized users.

Cortical coding

“…On the other hand, each neuron does respond very much more to some stimuli than to many others, and in this sense is tuned to some stimuli. This type of representation is not found in some models of invariant object representation in the visual cortical areas such as HMAX, but is approximated in VisNet (Robinson & Rolls, 2015). (HMAX C layer neurons are extremely broadly tuned (Robinson & Rolls, 2015).…”

“…(HMAX C layer neurons are extremely broadly tuned (Robinson & Rolls, 2015). When interpreted by a machine learning tool such as support vector machine learning, the representation that is required by the machine learning is just that specified by the experimenter, which might be for one object, or typically for any example of a single class of object such as hats or bears (Robinson & Rolls, 2015). )…”

Similarities and differences between stimulus tuning in the inferotemporal visual cortex and convolutional networks

“…CNNs can not be considered realistic models of the ventral stream, because they learn in very different ways, consist of much simpler units, typically lack lateral connections, etc. [11]. However, the fact that they have realistic core object recognition capabilities, and also exhibit internal representations that resemble those in the ventral stream, constrains ideas about the roles of these missing details in vision.…”

Spatial coordinate transforms linking the allocentric hippocampal and egocentric parietal primate brain systems for memory, action in space, and navigation

“…However, the results described here show that even if that were the case, then temporal trace learning would be very helpful in allowing neurons to achieve useful and accurate invariance. However, the concept in VisNet is that temporal trace learning is sufficient to learn transform invariant representations of objects in the ventral visual system, with the great advantage that this mechanism can also account for other forms of invariance, including view‐invariant representations, which cannot be learned by a spatial coordinate transform (as different views of a given object may be completely different), but which VisNet with its temporal short‐term memory trace approach learns well (Bart & Hegde, ; Perry, Rolls, & Stringer, ; Robinson & Rolls, ; Rolls, , ; Rolls & Mills, ; Rolls & Stringer, ; Rolls & Webb, ; Wallis & Rolls, ; Webb & Rolls, ; Zhao et al, ).…”