Exploring and explaining properties of motion processing in biological brains using a neural network

Abstract

Visual motion perception underpins behaviours ranging from navigation to depth perception and grasping. Our limited access to biological systems constrain our understanding of how motion is processed within the brain. Here we explore properties of motion perception in biological systems by training a neural network (‘MotionNetxy’) to estimate the velocity image sequences. The network recapitulates key characteristics of motion processing in biological brains, and we use our complete access to its structure explore and understand motion (mis)perception at the computational-, neural-, and perceptual-levels. First, we find that the network recapitulates the biological response to reverse-phi motion in terms of direction. We further find that it overestimates the speed of slow reverse-phi motion while underestimating the speed of fast reverse-phi motion because of the correlation between reverse-phi motion and the spatiotemporal receptive fields tuned to motion in opposite directions. Second, we find that the distribution of spatiotemporal tuning properties in the V1 and MT layers of the network are similar to those observed in biological systems. We then show that compared to MT units tuned to fast speeds, those tuned to slow speeds primarily receive input from V1 units tuned to high spatial frequency and low temporal frequency. Third, we find that there is a positive correlation between the pattern-motion and speed selectivity of MT units. Finally, we show that the network captures human underestimation of low coherence motion stimuli, and that this is due to pooling of noise and signal motion. These findings provide biologically plausible explanations for well-known phenomena, and produce concrete predictions for future psychophysical and neurophysiological experiments.

How multisensory neurons solve causal inference

“…Image translation, motion-in-depth, and rotation were performed in MATAB using Psychtoolbox version 3.0.11 subpixel rendering extensions ( 41 , 42 ) ( http://psychtoolbox.org/ ). The speeds used to train the network were selected because they did not exceed the image dimensions (64 × 64 pixels) and were similar to those used in our previous studies ( 15 , 16 ). We generated 64,000 motion sequences, which were scaled so that pixel intensities were between –1 and 1 and randomly divided into training and test sets, as described in the Training Procedure section.…”

“…Our complete access to the artificial system allows unrestricted interrogation of how the problem of causal inference is solved. We previously applied this method to understand unexplained features and reveal characteristics of biological motion processing in areas V1 and MT ( 15 , 16 ). Here, we apply the same principles to provide an explanation for how visual and vestibular sensory cues that signal motion are processed in MSTd to yield estimates of self- and scene motion and how causal inference is solved by biological systems.…”

Neural tuning instantiates prior expectations in the human visual system

“…Analogously, humans are superior on a range of visual tasks for stimuli that are oriented around cardinal orientations relative to oblique orientations 12 . Such biases in encoding and behaviour are even present in artificial intelligence systems trained on naturalistic movies 13,14 . Several computational accounts have attempted to unify the influence of environmental statistics on the properties of sensory neurons as well as perception 5,15,16 , but have been unable to address empirically how such encoding is implemented at the neural level.…”