Information limits on neural identification of colored surfaces in natural scenes

Abstract

If surfaces in a scene are to be distinguished by their color, their neural representation at some level should ideally vary little with the color of the illumination. Four possible neural codes were considered: von-Kries-scaled cone responses from single points in a scene, spatial ratios of cone responses produced by light reflected from pairs of points, and these quantities obtained with sharpened (opponent-cone) responses. The effectiveness of these codes in identifying surfaces was quantified by information-theoretic measures. Data were drawn from a sample of 25 rural and urban scenes imaged with a hyperspectral camera, which provided estimates of surface reflectance at 10-nm intervals at each of 1344 x 1024 pixels for each scene. In computer simulations, scenes were illuminated separately by daylights of correlated color temperatures 4000 K, 6500 K, and 25,000 K. Points were sampled randomly in each scene and identified according to each of the codes. It was found that the maximum information preserved under illuminant changes varied with the code, but for a particular code it was remarkably stable across the different scenes. The standard deviation over the 25 scenes was, on average, approximately 1 bit, suggesting that the neural coding of surface color can be optimized independent of location for any particular range of illuminants.

Visual sensitivity to color errors in images of natural scenes

“…Images were corrected for dark noise, spatial nonuniformities and stray light. For further details, see Foster et al~2004!. Nine scenes were selected from the database of images acquired in the Minho region of Portugal. The scenes represented rural and urban environments imaged at several distances, from near to far.…”

Compact spatio-spectral algorithm for single image super-resolution in hyperspectral imaging

“…The peak-transmission wavelength was varied in 10 nm steps over 400 −720 nm and the bandwidth was 10 nm at 510 nm, decreasing to 7 nm at 400 nm to 16 nm at 720 nm. The resulting test hyperspectral image x H has 1000 × 1000 pixels of spatial resolution and L = 32 spectral bands (Foster, 2004). …”

Recovering spectral data from natural scenes with an RGB digital camera and colored filters

“…The natural spectra were obtained by hyperspectral imaging 5 and the recovery matrix was estimated using the simulated responses of a conventional RGB digital camera coupled to several combinations of colored filters.…”

“…5 The scenes were taken from the Minho region of Portugal, which has a temperate climate and variety of vegetation and natural rock formations. The hyperspectral imaging system used a Peltier-cooled digital monochromatic camera with spatial resolution 1344 3 1024 pixels (Hamamatsu, model C4742-95-12ER, Hamamatsu Photonics K.K., Japan) with a fast-tunable liquid-crystal filter (VariSpec, model VS-VIS2-10-HC-35-SQ, Cambridge Research & Instrumentation Inc., Woburn, MA) mounted in front of the lens, with infra-red blocking filter.…”

“…[1][2][3][4] Typically, a multispectral system consists of a digital camera coupled to a range of spectrally broad-band or narrow-band filters. If the number of filters is sufficiently large and their bandwidths are sufficiently small, as with a hyperspectral imaging system, 2,5,6 spectral data can be recovered exactly. [5][6][7][8][9] But with just a few broad-band filters, spectral recovery presents an ill-posed problem.…”