Contrast sensitivity functions in autoencoders

Abstract

Three decades ago, Atick et al. suggested that human frequency sensitivity may emerge from the enhancement required for a more efficient analysis of retinal images. Here we reassess the relevance of low-level vision tasks in the explanation of the contrast sensitivity functions (CSFs) in light of 1) the current trend of using artificial neural networks for studying vision, and 2) the current knowledge of retinal image representations. As a first contribution, we show that a very popular type of convolutional neural networks (CNNs), called autoencoders, may develop human-like CSFs in the spatiotemporal and chromatic dimensions when trained to perform some basic low-level vision tasks (like retinal noise and optical blur removal), but not others (like chromatic) adaptation or pure reconstruction after simple bottlenecks). As an illustrative example, the best CNN (in the considered set of simple architectures for enhancement of the retinal signal) reproduces the CSFs with a root mean square error of 11% of the maximum sensitivity. As a second contribution, we provide experimental evidence of the fact that, for some functional goals (at low abstraction level), deeper CNNs that are better in reaching the quantitative goal are actually worse in replicating human-like phenomena (such as the CSFs). This low-level result (for the explored networks) is not necessarily in contradiction with other works that report advantages of deeper nets in modeling higher level vision goals. However, in line with a growing body of literature, our results suggests another word of caution about CNNs in vision science because the use of simplified units or unrealistic architectures in goal optimization may be a limitation for the modeling and understanding of human vision.

Contrast Sensitivity Function in Deep Networks

“…above 0.90 average correlation in area2 ). These findings match well previous reports explaining the relevance of low-level vision tasks in the emergence of human-like CSF (Li et al, 2022).…”

“…Features of other areas in denoising and autoencoding networks also match the shape of human CSF, for instance, the area3 of both networks or area1 in the denoising network. This is at odds with Li et al (2022) that reported human-like CSF only emerges in shallow CNNs optimised on similar tasks, and deeper CNNs fail to obtain human-like CSF although they reach the quantitative goal better than shallower networks. Our results suggest deeper networks of denoising and autoencoding also capture the human CSF.…”

“…These explanations of CSF suggest early visual processes are the basis of distinct visibility thresholds at different spatial frequencies; indeed, our ability to process contrast begins with single neurons in the retina and lateral geniculate nucleus excited most by light in the centre coinciding with dark in the surroundings or vice versa (Kuffler, 1953; Hubel and Wiesel, 1961). In the early visual cortex, neurons become sensitive to patterns of light that specify changes in orientation, spatial frequency, motion, and colour (Hubel and Wiesel, 2004) with highly localised responses strongly modulated by contrast; such single neurons are thought to underlie the human CSF and to efficiently code our environments (Atick et al, 1992; Atick and Redlich, 1992; Atick, 2011; Li et al, 2022). Although there is no doubt about the advantages of efficient visual processing (Geisler, 2008; Olshausen and Field, 1996), it is unclear how other important evolutionary behaviours (such as high-level visual tasks like object recognition) have affected the emergence of CSF in biological systems.…”

On the synthesis of visual illusions using deep generative models

“…This human-like behavior of CNNs trained for image restoration makes sense because the enhancement of the retinal signal may be one of the goals of the lateral geniculate nucleus ( Martinez-Otero et al, 2014 ). Moreover, other human-like features (e.g., contrast sensitivity) may emerge from this error minimization goal ( Gomez-Villa et al, 2020 ; Li et al, 2022 ). Analogous to the CVM case, we can measure the effect of the illusion by comparing the left and right targets in the image processed by the CNN.…”

Information Flow in Biological Networks for Color Vision

“…The conventional approach to check this hypothesis is from-statistics-to-perception : i.e., deriving the biological behavior from statistical arguments. In color vision, this includes the derivation of opponent channels from principal components [ 7 ], and the derivation of the frequency bandwidth of color channels [ 8 , 9 ] nonlinearities of opponent channels [ 10 , 11 ], and even the reproduction of color illusions [ 12 , 13 ], from information maximization or error minimization arguments. However, there is an alternative way to check the hypothesis: from-perception-to-statistics [ 14 , 15 , 16 ].…”