But Still It Moves: Static Image Statistics Underlie How We See Motion

Abstract

Seeing movement promotes survival. It results from an uncertain interplay between evolution and experience, making it hard to isolate the drivers of computational architectures found in brains. Here we seek insight into motion perception using a neural network (Motion-Net) trained on moving images to classify velocity. The network recapitulates key properties of motion direction and speed processing in biological brains, and we use it to derive, and test, understanding of motion (mis)perception at the computational, neural, and perceptual levels. We show that diverse motion characteristics are largely explained by the statistical structure of natural images, rather than motion per se. First, we show how neural and perceptual biases for particular motion directions can result from the orientation structure of natural images. Second, we demonstrate an interrelation between speed and direction preferences in (macaque) MT neurons that can be explained by image autocorrelation. Third, we show that natural image statistics mean that speed and image contrast are related quantities. Finally, using behavioral tests (humans, both sexes), we show that it is knowledge of the speed-contrast association that accounts for motion illusions, rather than the distribution of movements in the environment (the "slow world" prior) as premised by Bayesian accounts. Together, this provides an exposition of motion speed and direction estimation, and produces concrete predictions for future neurophysiological experiments. More broadly, we demonstrate the conceptual value of marrying artificial systems with biological characterization, moving beyond "black box" reproduction of an architecture to advance understanding of complex systems, such as the brain.

Exploring and explaining properties of motion processing in biological brains using a neural network

“…3d, cyan markers). This pattern of responses is consistent with our previous work (Rideaux & Welchman, 2020), where we showed that low speed motion sequences moving in different directions are highly correlated, thus directions are less distinguishable than high speed sequences.…”

“…We previously showed a similar pattern of selectivity emerged in a neural network ("MotionNet") trained to make discrete velocity classifications (Rideaux & Welchman, 2020); however, these results differed from biological findings in that MT units were exclusively pattern-motion selective (rather than containing a mixture of selectivity; Fig. 2d).…”

“…Image translation was performed in MATAB using Psychtoolbox v3.0.11 subpixel rendering extensions (Brainard, 1997;Pelli, 1997) (http://psychtoolbox.org/). The speeds used to train the network were selected because they did not exceed the image dimensions (32 × 32 pixels) and matched those used in our previous study (Rideaux & Welchman, 2020). We generated 32,000 motion sequences, which were scaled so that pixel intensities were between -1 and 1, and randomly divided into training and test sets, as described in the Training Procedure section.…”

“…To compare the distribution of pattern-motion selectivity among V1 and MT units in MotionNetxy with those of our previous network ("MotionNet"; Rideaux & Welchman, 2020) and V1 and MT neurons, we projected the values shown in Figure 2b and Figure 2c, in addition to data from Figure 3e our previous study (Rideaux & Welchman, 2020) along the diagonal to establish a unified estimate of pattern-motion selectivity for each unit (Fig. 2d-f).…”

Biased orientation representations can be explained by experience with non-uniform training set statistics

“…First, our results show that a popular CNN model exhibits a form of the classical oblique effect, suggesting that this key aspect of low-level primate vision is reproduced by the model. This adds to a growing body of work demonstrating similarities between deep neural networks and the brains and behavior of primates (28)(29)(30)(31)(32). Second, we have demonstrated that nonuniformities in the statistics of training set images can dramatically influence the feature representations that are learned by a CNN.…”