Mondrian Representation of Real World Scene Statistics

Abstract

In material appearance we are interested in objectively measuring a physical aspect of a material, such as reflectance, and also in understanding how we see that material. In perceptual experiments we typically display simple stimuli to an observer, record their response, and then try to build a theory of why the observer responded in a given way. Often the latter model is implemented as a computer algorithm, and many of these are, for example, now implemented in camera pipelines for smartphones. However, the stimuli that are shown to observers are necessarily either very simple, such as rectangular patches of colour, or small in number. This raises the question as to whether the recorded responses to simple stimuli actually shed light on how we perceive scenes in the real world.In this paper, we look at a specific example of perceptual stimuli: Mondrian images, and investigate the extent to which, in the sense of their autocorrelation matrix, they represent the real world. We show that by modelling paths of pixels through an image using a statistical model that captures the statistics of real Mondrians, the autocorrelation matrix of Mondrians is Toeplitz, and moreover this Toeplitz structure is also found in real images. Although Mondrian images do not contain typical visual cues, our path model can be tuned to replicate the statistics of real images in the autocorrelation sense. The practical utility of this method is that paths through images and their autocorrelation statistics are a key tool for developing algorithms to predict the perceptual response to complex scenes. For example, this approach is at the foundation of retinex image processing. Experiments validate our method.