Deep Neural Networks for Modeling Visual Perceptual Learning

Abstract

Understanding visual perceptual learning (VPL) has become increasingly more challenging as new phenomena are discovered with novel stimuli and training paradigms. Although existing models aid our knowledge of critical aspects of VPL, the connections shown by these models between behavioral learning and plasticity across different brain areas are typically superficial. Most models explain VPL as readout from simple perceptual representations to decision areas and are not easily adaptable to explain new findings. Here, we show that a well -known instance of deep neural network (DNN), whereas not designed specifically for VPL, provides a computational model of VPL with enough complexity to be studied at many levels of analyses. After learning a Gabor orientation discrimination task, the DNN model reproduced key behavioral results, including increasing specificity with higher task precision, and also suggested that learning precise discriminations could transfer asymmetrically to coarse discriminations when the stimulus conditions varied. Consistent with the behavioral findings, the distribution of plasticity moved toward lower layers when task precision increased and this distribution was also modulated by tasks with different stimulus types. Furthermore, learning in the network units demonstrated close resemblance to extant electrophysiological recordings in monkey visual areas. Altogether, the DNN fulfilled predictions of existing theories regarding specificity and plasticity and reproduced findings of tuning changes in neurons of the primate visual areas. Although the comparisons were mostly qualitative, the DNN provides a new method of studying VPL, can serve as a test bed for theories, and assists in generating predictions for physiological investigations. Visual perceptual learning (VPL) has been found to cause changes at multiple stages of the visual hierarchy. We found that training a deep neural network (DNN) on an orientation discrimination task produced behavioral and physiological patterns similar to those found in human and monkey experiments. Unlike existing VPL models, the DNN was pre-trained on natural images to reach high performance in object recognition, but was not designed specifically for VPL; however, it fulfilled predictions of existing theories regarding specificity and plasticity and reproduced findings of tuning changes in neurons of the primate visual areas. When used with care, this unbiased and deep-hierarchical model can provide new ways of studying VPL from behavior to physiology.

Interfering with a memory without erasing its trace

“…In a more complicated read-out scenario with multiple read-out units that do not share feedforward weights, no interference at all is expected, which also conflicts with our empirical data. Notably, it has recently been shown that exposing a purely feedforward convolutional neural network to orientation discrimination training can sharpen tuning curves in early layers of the model (reflecting V1-and V2-like processing; Wenliang & Seitz, 2018). The convolutional neural network has, however, not been used to investigate interference.…”

Can Deep Learning Model Perceptual Learning?

“…A method that can potentially differentiate between the two models is an inactivation experiment. In a recent study, Liu and Pack (2017) showed that inactivation of middle temporal area (MT) in the visual dorsal pathway influenced motion perception thresholds only after a monkey was trained on a stimulus that better stimulated MT (Liu and Pack, 2017). This training, according to the reweighting model, increases the readout weight of MT neurons as the most informative neurons for the task.…”

“…Therefore, in the architecture used by Wenliang and Seitz (2018), the DNN is not able to explain this inactivation result. However, in an extended DNN with skip connections, the inactivation experiment can be simulated and compared with the empirical findings (Liu and Pack, 2017).…”

Can multisensory training aid visual learning? A computational investigation

“…Similarly, many artificial neural networks lack biological detail, resembling biological neural networks only in seemingly coarsescale ways. Despite this, several researchers have recently argued that these networks provide useful insights into neural processing, particularly within the visual system (Kriegeskorte, 2015;Wenliang & Seitz, 2018;Yamins & DiCarlo, 2016). Readers interested in the relationships between artificial and biological neural networks may see Churchland and Sejnowski (2017), Kriegeskorte and Golan (2019), Marblestone, Wayne, and Kording (2016), and Yamins and DiCarlo (2016).…”

“…First, visual features in the models were extracted using a network with alternating layers of convolutional and pooling units (followed by fully connected layers) whose processing is reminiscent of processing in visual cortical areas. Previous researchers have found that processing in networks using convolution and pooling resembles biological visual processing in intriguing ways (Kriegeskorte, 2015;Wenliang & Seitz, 2018;Yamins & DiCarlo, 2016). Second, the models make explicit use of efficient data compression to learn compact representations of perceptual data items.…”