Perceptually based tone mapping for low-light conditions

Abstract

We present a perceptually based algorithm for modeling the color shift that occurs for human viewers in low-light scenes. Known as the Purkinje effect, this color shift occurs as the eye transitions from photopic, cone-mediated vision in well-lit scenes to scotopic, rod-mediated vision in dark scenes. At intermediate light levels vision is mesopic with both the rods and cones active. Although the rods have a spectral response distinct from the cones, they still share the same neural pathways. As light levels decrease and the rods become increasingly active they cause a perceived shift in color. We model this process so that we can compute perceived colors for mesopic and scotopic scenes from spectral data. While our tone mapping operator works with spectral data from any source, we show how to produce spectral data of static scenes using multiple images and a camera with known spectral sensitivity. If the spectral sensitivity isn't provided by the camera manufacturer, we describe a one-time calibration procedure to estimate the sensitivity. Should obtaining spectral data of a scene be infeasible, we also describe how the effect can be approximated from high dynamic range RGB images by learning a mapping from RGB to rod and cone responses. Once we have determined rod and cone responses either directly or through our approximation process, we map them to RGB values that can be displayed on a standard monitor to elicit the same color perception when viewed photopically. Our tone mapping method focuses on computing the color shift associated with low-light conditions and leverages current HDR techniques to control the image dynamic range. We include results generated from both spectral and RGB images as well as experimental data.

Practical Capture and Reproduction of Phosphorescent Appearance

“…Professional spectrometers are expensive. An alternative is taking RGB images of the respective sample using a camera with known spectral sensitivity, through a number of cheap color filters with different, known transmission curves, and solving the resulting system of linear equations to find the spectrum best explaining the measured values in a least-squares sense [KO11]. Unfortunately, phosphorescence is a time-varying effect requiring to take these images at the same instant, which is hard to achieve with common filters.…”

Modeling Luminance Perception at Absolute Threshold

“…Tone mapping: Night scene depiction A key goal of tone mapping operators (TMO) is to reproduce scotopic scene appearance on a photopic display [RWD*10] by simulating a blue‐shift and the loss of color vision, visual acuity, contrast and brightness characteristic for night vision [FPSG96, PFFG98, KP04, KO11, WM14]. Typically, such simulations cover higher levels of scotopic luminance (0.001–0.1 cd / m 2 ) including the transition to mesopic conditions, while luminance levels near absolute thresholds are not specifically addressed.…”

“…Computer graphics has now routinely modeled the shift from photopic over in‐between mesopic to scotopic conditions [FPSG96, PFFG98, DD00, PTYG00, TSF02, KP04, KO11, WM14] but the scotopic regime (vision close to its absolute threshold, e. g., a moonless night), has received little attention. Remarkably, the absolute threshold is close to the physical limitations of light itself; most dark‐adapted subjects reliably detect flashes of light resulting in as little as 5 to 10 photons total on the retina during an integration time of 100ms [HSP42].…”

“…Their peak density is lower than for cones, but their distribution over the retina is more uniform and shows a slower falloff with the retinal eccentricity. Modelling these differences is the foundation of many day‐for‐night tone mappers [DD00, PTYG00, TSF02, KP04, KO11, WM14].…”

Purkinje Images: Conveying Different Content for Different Luminance Adaptations in a Single Image

“…A linear combination of, possibly differently derived, rod and cone responses is typically used in image quality metrics [MKRH11], colour appearance models [RKAJ08] and tone mapping operators [FPSG96, DD00, WRP97]. Remarkable realism of tone mapping for spectral images has been shown by Kirk and O'Brien [KO11], who employed a biologically inspired model that predicts the offset in the L, M and S-cone channels due to rod response [CPSZ08]. An advanced colour appearance and tone mapping approach in a single framework has been proposed by Pattanaik et al [PFFG98], where different contrast transducers are considered for cone-and rod-mediated signals, prior to their combination into an achromatic signal.…”