Apparent display resolution enhancement for moving images

Abstract

Figure 1: Depicting fine details such as hair (left), sparkling car paint (middle) or small text (right) on a typical display is challenging and often fails if the display resolution is insufficient. In this work, we show that smooth and continuous subpixel image motion can be used to increase the perceived resolution. By sequentially displaying varying intermediate images at the display resolution (as depicted in the bottom insets), subpixel details can be resolved at the retina in the region of interest due to fixational eye tracking of this region. AbstractLimited spatial resolution of current displays makes the depiction of very fine spatial details difficult. This work proposes a novel method applied to moving images that takes into account the human visual system and leads to an improved perception of such details. To this end, we display images rapidly varying over time along a given trajectory on a high refresh rate display. Due to the retinal integration time the information is fused and yields apparent super-resolution pixels on a conventional-resolution display. We discuss how to find optimal temporal pixel variations based on linear eye-movement and image content and extend our solution to arbitrary trajectories. This step involves an efficient method to predict and successfully treat potentially visible flickering. Finally, we evaluate the resolution enhancement in a perceptual study that shows that significant improvements can be achieved both for computer generated images and photographs.

Apparent resolution enhancement for animations

“…Recently, it was shown by Didyk et al [2010], how taking into account the HVS's SPEM, as well as its temporally integrating nature, the apparent resolution of a display can be enhanced for moving images. They proposed a simplified model of the temporal integration on the human retina, which allows them to predict the perceived image.…”

“…They also propose a small circular path for the motion as the best choice, which at higher frame-rates can lead to simple averaging and no resolution gain. Whereas Didyk et al [2010] introduced a motion for images in order to increase the apparent resolution, we show how to use existing motion in the scene (e. g., rendering, video animation) to achieve an enhancement as well. Therefore we will mostly rely on their findings extending them to arbitrary motion in a scene.…”

“…Therefore we will mostly rely on their findings extending them to arbitrary motion in a scene. Didyk et al [2010] assume that after time T the response of a single receptor r moving over an image I is given by the following equation:…”

“…While standard down-sampling methods treat each frame of an animation separately without taking temporal effects into account, it was shown by Didyk et al [2010], how properties of the HVS can be beneficially exploited for moving images. The smooth pursuit eye movement (SPEM) mechanism can be used to induce subpixel shifts of the image projected on the retina.…”

Stereoscopic 3D display technique using spatiotemporal interlacing has improved spatial and temporal properties

“…When the stimulus moves horizontally, the missing data are not presented at any time, so performance does not improve. The use of motion to create higher effective resolution has been examined extensively in computer graphics [25]. We conclude that the single-frame version of the hybrid protocol has better spatial resolution than the spatial-interlacing protocol.…”