Modeling second-order boundary perception: A machine learning approach

DiMattina, Christopher; Baker, Curtis L.

doi:10.1371/journal.pcbi.1006829

Cited by 18 publications

(20 citation statements)

References 116 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found that many units were identified as blur selective across all levels of the network and we defined a class of units that we refer to as blur-contrast boundary (BCB) units that respond best to oriented or shaped boundaries between regions of high and low blur levels. Our results raise the testable prediction that similar units may be prevalent in vivo, related to second-order boundary processing (Mareschal and Baker, 1998;Dakin and Mareschal, 2000;Baker and Mareschal, 2001;DiMattina and Baker, 2019), and they provide insight into improving simple stimulus sets for testing for blur selectivity.…”

Section: Introductionsupporting

confidence: 66%

Artiphysiology reveals visual preferences underlying V4-like blur selectivity in a deep convolutional neural network

Entezari

Bair

2019

Preprint

View full text Add to dashboard Cite

Blurry visual scenes arise from many causes and image blur is known to be important for scene interpretation, yet there have been very few studies of the visual encoding of blur in biological visual systems or in artificial visual systems trained for object recognition. Recently, a study of single neurons in the visual cortex of macaque monkeys found that a significant fraction of neurons were more responsive to blurred visual stimuli than to sharply defined stimuli. This raises two questions: (1) what types of visual features in natural scenes might underlie blur selectivity in macaque cortex, and (2) can blur selectivity be found in artificial neural networks using the simple artificial stimuli previously applied in vivo? To answer these questions, we presented simple shape stimuli to the widely studied deep convolutional neural network (CNN) known as AlexNet and used deconv-net visualization to identify features that are critical for driving blur selective units. We found that a substantial number of units in the CNN were selective for blur and that several categories of blur selectivity emerged in early-to-middle processing stages. Prominent among these is a set of units selective for spatial boundaries defined by blur contrast. These blur-contrast boundary units may serve the important task of segmentation in natural photographic images and may relate to a large body of literature on second-order boundary detection. Our results lead to the prediction that such units could exist in the visual cortex but have yet to be well characterized and localized, and they provide direction for future neurophysiological tests of blur selectivity in vivo.

show abstract

Section: Introductionsupporting

confidence: 66%

Artiphysiology reveals visual preferences underlying V4-like blur selectivity in a deep convolutional neural network

Entezari

Bair

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…We define and fit a "filterrectify-filter" (FRF) model positing two stages of filtering to data from Experiment 4, and show that this model successfully accounts for observer performance in the task. Previous studies of second-order vision have fit psychophysical data with FRF models (DiMattina & Baker, 2019;Zavitz & Baker, 2013, but here we show that the FRF model can also account for the ability of observers to extract first-order (luminance) information in the presence of masking LSB stimuli.…”

supporting

confidence: 46%

“…Although natural surfaces may have luminance differences which arise due to luminance texture boundaries, many other textural differences do not involve changes in luminance. Micro-pattern orientation, density, and contrast and others all provide powerful segmentation cues (Dakin & Mareschal, 2000; DiMattina & Baker, 2019; Zavitz & Baker, 2013, 2014; Wolfson & Landy, 1995; Motoyoshi & Kingdom, 2007), which must be combined with luminance cues to enable segmentation in natural vision. It is of great interest for future research to understand how luminance textures combine with other cues.…”

Section: Discussionmentioning

confidence: 99%

“…A number of studies have investigated detection of “first-order” luminance step boundaries (Elder & Sachs, 2004; McIlhagga & May, 2012; McIlhagga, 2018; McIlhagga & Mullen, 2018), as well as detection and segmentation of “second-order” texture boundaries having no luminance difference but differences in texture contrast (Dakin & Mareschal, 2000; DiMattina & Baker, 2019), density (Zavitz & Baker, 2014), orientation (Wolfson & Landy, 1995), polarity (Motoyoshi & Kingdom, 2007) or phase (Hansen & Hess, 2006). However, the segmentation of first-order luminance texture boundaries, and the underlying computations, are poorly understood.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Segmenting surface boundaries using luminance cues: Underlying mechanisms

DiMattina

Baker

2020

Preprint

Self Cite

View full text Add to dashboard Cite

ABSTRACTSegmenting the visual scene into distinct surfaces is one of the most basic tasks of visual perception, and luminance differences between two adjacent surfaces often provide an important segmentation cue. However, mean luminance differences between two surfaces may exist without any sharp change in albedo at their boundary, but rather from differences in the proportion of light and dark areas within each surface, which we refer to as a luminance texture boundary. In this study, we investigate the performance of human observers segmenting luminance texture boundaries, with the ultimate goal of understanding the underlying visual computations. We demonstrate that a simple model involving a single stage of filtering is inadequate to explain observer performance when segmenting luminance texture boundaries, but performance can be explained when contrast normalization is added to this one-stage model. Performing additional experiments in which observers segment luminance texture boundaries while ignoring super-imposed luminance step boundaries, we demonstrate that the one-stage model cannot explain segmentation performance in this task. We then present a Filter-Rectify-Filter (FRF) model positing two cascaded stages of filtering, and demonstrate that it neatly fits our data and explains our ability to segment luminance texture boundary stimuli in the presence of interfering luminance step boundaries, as well as in the absence of masking stimuli. We propose that such computations may be useful for boundary segmentation in natural scenes, where shadows often give rise to luminance step edges which do not correspond to surface boundaries. Finally, we suggest directions for future work to further investigate possible neural implementation of the computations suggested by our psychophysical study.

show abstract

“…Natural texture boundaries will not only contain differences in luminance, but also differences in various second-order texture cues like orientation (Wolfson & Landy, 1998), micro-pattern density (Zavitz & Baker, 2014), and contrast (Dakin & Mareschal, 2000;DiMattina & Baker, 2019). It is of interest for future work to better understand how luminance cues (step and boundary) interact with these various second-order texture cues for boundary segmentation.…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

Luminance texture boundaries and luminance step boundaries are segmented using different mechanisms

DiMattina

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In natural scenes, two adjacent surfaces may differ in mean luminance without any sharp change in luminance at their boundary, but rather due to different relative proportions of light and dark regions within each surface. We refer to such boundaries as luminance texture boundaries (LTBs), and in this study we investigate interactions between luminance texture boundaries and luminance step boundaries (LSBs) in a segmentation task. Using a simple masking paradigm, we find very little influence of LSB maskers on LTB segmentation thresholds. Similarly, we find only modest effects of LTB maskers on LSB thresholds. By contrast, each kind of boundary strongly masks targets of the same kind. Our data is consistent with the possibility that luminance texture boundaries may be segmented using different mechanisms than those used to segment luminance step boundaries. At the same time, our work also suggests that LTB segmentation is subject to influences from LSBs. We suggest that the relative robustness of LTB segmentation to interference from LSBs may serve the ecologically important role of providing robustness to changes in luminance caused by cast shadows, and we propose future experimental work to investigate this hypothesis.

show abstract

Modeling second-order boundary perception: A machine learning approach

Cited by 18 publications

References 116 publications

Artiphysiology reveals visual preferences underlying V4-like blur selectivity in a deep convolutional neural network

Artiphysiology reveals visual preferences underlying V4-like blur selectivity in a deep convolutional neural network

Segmenting surface boundaries using luminance cues: Underlying mechanisms

Luminance texture boundaries and luminance step boundaries are segmented using different mechanisms

Contact Info

Product

Resources

About