Modeling Natural Images Using Gated MRFs

Ranzato, Marc’Aurelio; Mnih, Volodymyr; Susskind, Joshua M.; Hinton, Geoffrey E.

doi:10.1109/tpami.2013.29

Cited by 61 publications

(34 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We reasoned that since V2 representation is more abstract and invariant, it does not have the precision to synthesize a high-resolution image to feed back to V1. Rather, feedback provides a set of global beliefs from a higher order extrastriate cortex as high level commands and instructions to condition and “work with” V1’s own intrinsic circuitry to synthesize a precise internal representation of the prediction of the input to V1 (see also [41]. A spatially fuzzy top-down feedback could work with the intrinsic circuitry in V1 to produce a spatially precise surface and figure-ground representations in V1.…”

Section: Varieties Of Internal Modelsmentioning

confidence: 99%

The Visual System's Internal Model of the World

Lee

2015

Proc. IEEE

View full text Add to dashboard Cite

The Bayesian paradigm has provided a useful conceptual theory for understanding perceptual computation in the brain. While the detailed neural mechanisms of Bayesian inference are not fully understood, recent computational and neurophysiological works have illuminated the underlying computational principles and representational architecture. The fundamental insights are that the visual system is organized as a modular hierarchy to encode an internal model of the world, and that perception is realized by statistical inference based on such internal model. In this paper, I will discuss and analyze the varieties of representational schemes of these internal models and how they might be used to perform learning and inference. I will argue for a unified theoretical framework for relating the internal models to the observed neural phenomena and mechanisms in the visual cortex.

show abstract

Section: Varieties Of Internal Modelsmentioning

confidence: 99%

The Visual System's Internal Model of the World

Lee

2015

Proc. IEEE

View full text Add to dashboard Cite

show abstract

“…Finding such a layered Ising model will be of major value for physics and computer science. It may also be relevant in neuroscience because it suggests a neural architecture in the brain for generating images (6,18).…”

Section: Discussionmentioning

confidence: 99%

Hierarchical model of natural images and the origin of scale invariance

Saremi

Sejnowski

2013

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The study of natural images and how our brain processes them has been an area of intense research in neuroscience, psychology, and computer science. We introduced a unique approach to studying natural images by decomposing images into a hierarchy of layers at different logarithmic intensity scales and mapping them to a quasi-2D magnet. The layers were in different phases: "cold" and ordered at large-intensity scales, "hot" and disordered at small-intensity scales, and going through a second-order phase transition at intermediate scales. There was a single "critical" layer in the hierarchy that exhibited long-range correlation similar to that found in the 2D Ising model of ferromagnetism at the critical temperature. We also determined the interactions between layers mapped from natural images and found mutual inhibition that generated locally "frustrated" antiferromagnetic states. Almost all information in natural images was concentrated in a few layers near the phase transition, which has biological implications and also points to the hierarchical origin of scale invariance in natural images.critical point | vision | generative models O ur visual system evolved to survive in nature with scenes of mountains, rivers, trees, and other animals (1). The neural representations of visual inputs are related to their statistical structure (1-3). Structures in nature come in a hierarchy of sizes that cannot be separated, a signature of scale invariance, which also occurs near a critical point in many physical systems. The classic example of a critical point is a uniaxial ferromagnetic system going through a second-order phase transition in a zero magnetic field by increasing temperature. At the critical point, the system loses its magnetization due to thermal fluctuations. There are large regions ("islands") that are magnetized in one direction but are surrounded by large regions ("seas") that are magnetized in the opposite direction. The seas themselves are embedded in bigger islands, ad infinitum. The total magnetization is zero, but the correlation length diverges, which is visualized by growth of the sizes of seas and islands with the system size. At the critical point, the system is free of a length scale because fluctuations occur at scales of all lengths. The infinite correlation length is thus intricately linked with scale invariance. The scale invariance in natural images was first characterized by the 1/f 2 spatial power spectrum of pixel intensities (2). Here, we study scaling properties of natural images at a deeper level by finding a hierarchy of statistical structures, in which the scale invariance emerges near a second-order phase transition.Images are preprocessed in the retina by a complex network with ∼55 distinct cell types in mammals (5). The cerebral cortex receives a spatiotemporal stream of spikes that contain all the information in the visual inputs that has been coded by the retina. Understanding the hierarchies of statistical structures in natural images is essential for better understanding how ...

show abstract

“…Note that the conclusions of [15] depend on the use of overlapping patches, while we propose a scheme that can operate efficiently on whole images and avoids stitching artifacts. A similar choice has been made, independently and concurrently, by [28][29][30][31]. The use of overlapping patches introduces unwanted redundancies, which, as will be discussed below, explains some of the apparent discrepancies between the outcome of the earlier study [15] and ours.…”

Section: Related Workmentioning

confidence: 96%

On the relevance of sparsity for image classification

Rigamonti

Lepetit

González

et al. 2014

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Abstract. In this paper we analyze empirically the importance of sparsifying representations for classification purposes. We focus on those obtained by convolving images with linear filters, which can be either hand designed or learned, and perform extensive experiments on two important Computer Vision problems, image categorization and pixel classification. To this end, we adopt a simple modular architecture that encompasses many recently proposed models. The key outcome of our investigations is that enforcing sparsity constraints on features extracted in a convolutional architecture does not improve classification performance, whereas it does so when redundancy is artificially introduced. This is very relevant for practical purposes, since it implies that the expensive run-time optimization required to sparsify the representation is not always justified, and therefore that computational costs can be drastically reduced.

show abstract

Modeling Natural Images Using Gated MRFs

Cited by 61 publications

References 54 publications

The Visual System's Internal Model of the World

The Visual System's Internal Model of the World

Hierarchical model of natural images and the origin of scale invariance

On the relevance of sparsity for image classification

Contact Info

Product

Resources

About