Curve Detectors

Cammarata, Nick; Goh, Gabriel; Carter, Shan; Schubert, Ludwig; Петров, М. Н.; Olah, Chris

doi:10.23915/distill.00024.003

Cited by 18 publications

(18 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is consistent with a growing body of evidence from the ventral stream (Bao et al, 2020;Rajalingham and DiCarlo, 2019;Yue et al, 2014). Recent explorations of neuronal response fields in artificial neural networks have likewise found a prevalence of curve detectors with increasing complexity along the processing hierarchy (Cammarata et al, 2020). Studying such functional cross-areal connectivity (both bottom-up and topdown) remains a critical goal for future studies of the visual system.…”

Section: Discussionsupporting

confidence: 77%

Clustered functional domains for curves and corners in cortical area V4

Jiang

Andolina

et al. 2021

eLife

View full text Add to dashboard Cite

The ventral visual pathway is crucially involved in integrating low-level visual features into complex representations for objects and scenes. At an intermediate stage of the ventral visual pathway, V4 plays a crucial role in supporting this transformation. Many V4 neurons are selective for shape segments like curves and corners, however it remains unclear whether these neurons are organized into clustered functional domains, a structural motif common across other visual cortices. Using two-photon calcium imaging in awake macaques, we confirmed and localized cortical domains selective for curves or corners in V4. Single-cell resolution imaging confirmed that curve or corner selective neurons were spatially clustered into such domains. When tested with hexagonal-segment stimuli, we find that stimulus smoothness is the cardinal difference between curve and corner selectivity in V4. Combining cortical population responses with single neuron analysis, our results reveal that curves and corners are encoded by neurons clustered into functional domains in V4. This functionally-specific population architecture bridges the gap between the early and late cortices of the ventral pathway and may serve to facilitate complex object recognition.

show abstract

Section: Discussionsupporting

confidence: 77%

Clustered functional domains for curves and corners in cortical area V4

Jiang

Andolina

et al. 2021

eLife

View full text Add to dashboard Cite

show abstract

“…First, training led to an increase in representational similarities in early and intermediate layers, indicating that learned features support the representational similarities between types of depiction. These features may include local edge features in early convolutional layers ( Krizhevsky et al, 2012 ) or curvature features in intermediate layers ( Cammarata, Goh, Carter, Schubert, Petrov, & Olah, 2020 ). In contrast to these results in early layers, training decreased the representational similarities in later layers, likely reflecting the bias of the network for the statistics of natural images, which is found in the photos but neither the drawings nor the sketches.…”

Section: Discussionmentioning

confidence: 99%

From photos to sketches - how humans and deep neural networks process objects across different levels of visual abstraction

Singer

Seeliger

Kietzmann³

et al. 2022

Journal of Vision

View full text Add to dashboard Cite

Line drawings convey meaning with just a few strokes. Despite strong simplifications, humans can recognize objects depicted in such abstracted images without effort. To what degree do deep convolutional neural networks (CNNs) mirror this human ability to generalize to abstracted object images? While CNNs trained on natural images have been shown to exhibit poor classification performance on drawings, other work has demonstrated highly similar latent representations in the networks for abstracted and natural images. Here, we address these seemingly conflicting findings by analyzing the activation patterns of a CNN trained on natural images across a set of photographs, drawings, and sketches of the same objects and comparing them to human behavior. We find a highly similar representational structure across levels of visual abstraction in early and intermediate layers of the network. This similarity, however, does not translate to later stages in the network, resulting in low classification performance for drawings and sketches. We identified that texture bias in CNNs contributes to the dissimilar representational structure in late layers and the poor performance on drawings. Finally, by fine-tuning late network layers with object drawings, we show that performance can be largely restored, demonstrating the general utility of features learned on natural images in early and intermediate layers for the recognition of drawings. In conclusion, generalization to abstracted images, such as drawings, seems to be an emergent property of CNNs trained on natural images, which is, however, suppressed by domain-related biases that arise during later processing stages in the network.

show abstract

“…An extensive analysis of features, connections, and their organization extracted from trained Incep-tionV1 [10] was presented in [11][12][13][14][15][16][17][18][19]. The authors of [20] studied learned filter representations in ImageNet classification models and presented the first moves towards transfer learning.…”

Section: Related Workmentioning

confidence: 99%

An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters

Gavrikov¹,

Keuper²

2022

Preprint

View full text Add to dashboard Cite

We present first empirical results from our ongoing investigation of distribution shifts in image data used for various computer vision tasks. Instead of analyzing the original training and test data, we propose to study shifts in the learned weights of trained models. In this work, we focus on the properties of the distributions of dominantly used 3 × 3 convolution filter kernels. We collected and publicly provide a data set with over half a billion filters from hundreds of trained CNNs, using a wide range of data sets, architectures, and vision tasks. Our analysis shows interesting distribution shifts (or the lack thereof) between trained filters along different axes of meta-parameters, like data type, task, architecture, or layer depth. We argue, that the observed properties are a valuable source for further investigation into a better understanding of the impact of shifts in the input data to the generalization abilities of CNN models and novel methods for more robust transfer-learning in this domain. Data available at: https://github.com/paulgavrikov/CNN-Filter-DB/. IntroductionDespite their overwhelming success in the application to various vision tasks, the practical deployment of convolutional neural networks (CNNs) is still suffering from several inherent drawbacks. Two prominent examples are I) the dependence on very large amounts of annotated training data [1], which is not available for all target domains and is expensive to generate; and II) still widely unsolved problems with the robustness and generalization abilities of CNNs [2] towards shifts of the input data distributions. One can argue that both problems are strongly related, since a common practical solution to I) is the fine-tuning [3] of pre-trained models by small data sets from the actual target domain. This results in the challenge to find suitable pre-trained models based on data distributions that are "as close as possible" to the target distributions. Hence, both cases (I+II) imply the need to model and observe distribution shifts in the contexts of CNNs. In this paper, we propose not to investigate these shifts in the input (image) domain, but rather in the weight distributions of the CNNs themselves. We argue that e.g. the distributions of trained convolutional filters in a CNN, which implicitly reflect the sub-distributions of the input image data which are actually utilized by a specific model, are more suitable and easier accessible representations for this task. MethodsData. We collected a total of 391 publicly available CNN models pre-trained for various visual tasks, recorded meta-data for each model, and manually categorized the training data into visually distinctive groups (data type) like natural scenes, medical ct, seismic, or astronomy for example. All models were trained with full 32-bit precision but may have been trained with variously scaled inputs. The dominant subset is formed by image classification models trained on ImageNet1k [4] (264 models). We extracted all trained convolution filters to get a heterogeneous a...

show abstract

Curve Detectors

Cited by 18 publications

References 9 publications

Clustered functional domains for curves and corners in cortical area V4

Clustered functional domains for curves and corners in cortical area V4

From photos to sketches - how humans and deep neural networks process objects across different levels of visual abstraction

An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters

Contact Info

Product

Resources

About