From information scaling of natural images to regimes of statistical models

Abstract

Abstract. Vision can be considered a highly specialized data collection and analysis problem. We need to understand the special properties of natural image data in order to construct statistical models and develop statistical methods for representing and recognizing the wide variety of natural image patterns. One fundamental property of natural image data that distinguishes vision from other sensory tasks such as speech recognition is that scale plays a profound role in image formation and interpretation. Specifically, visual objects can appear at a wide range of scales in the images due to the change of viewing distance as well as camera resolution. The same objects appearing at different scales produce different image data with different statistical properties. In particular, we show that the entropy rate of the image data changes over scale. Moreover, the inferential uncertainty changes over scale too. We call these changes information scaling. We then examine both empirically and theoretically two prominent and yet largely isolated classes of image models, namely, wavelet sparse coding models and Markov random field models. Our results indicate that the two classes of models are appropriate for two different entropy regimes: sparse coding targets low entropy regimes, whereas Markov random fields are appropriate for high entropy regimes. Because information scaling connects different entropy regimes, both sparse coding and Markov random fields are necessary for representing natural image data, and information scaling triggers transitions between these two regimes. This motivates us to propose a modeling scheme that embraces both regimes of models in a common framework. The contribution of our work is two-fold. First, the study of information scaling provides a unifying perspective for the rich variety of natural image patterns. Second, the modeling scheme that we develop provides a natural integration of different regimes of image models.

Actionable Information in Vision

“…However, the rendition of the "primal sketch" [54] in [35] does not guarantee that the construction is "lossless" with respect to any particular task, because there is no underlying task guiding the construction. Our work also relates to the vast literature on segmentation, particularly texture-structure transitions [83]. Alternative approaches to this task could be specified in terms of sparse coding [59] and non-local filtering [15].…”

Learning Hybrid Image Templates (HIT) by Information Projection

“…In object recognition, sketch features are shown to work well on objects with regular shapes, while texture features are more suitable for complex objects with cluttered appearance. These two types of features are often studied separately for structures at different resolutions, but in real images, they are connected continuously through image scaling [8]. That is, viewed in low resolution, geometric structures become blurred and merge into texture appearance, and can become flat area (white noise) at extremely low resolution.…”

Unsupervised learning of compositional sparse code for natural image representation

“…First, as illustrated by Figure 19, as we zoom out the images, the image patterns undergo a transition from low-entropy regime of geometric structures to mid-entropy regime of object shapes to high-entropy regime of stochastic textures [36] (patterns such as the brick wall are also textures, but they for the distribution of the nominal template (before shape deformation), where λ x,s,α can be either positive (for sketch patterns) or negative (for flatness patterns). One can sparsify λ x,s,α by 1 penalized maximum likelihood.…”

“…Information theoretical interpretation. The exponential family model can be justified by the maximum entropy principle [16,28,36,43]. Given the deformed template…”