Programmable imaging using a digital micromirror array

Abstract

In this paper, we introduce the notion of a programmable imaging system. Such an imaging system provides a human user or a vision system significant control over the radiometric and geometric characteristics of the system. This flexibility is achieved using a programmable array of micro-mirrors. The orientations of the mirrors of the array can be controlled with high precision over space and time. This enables the system to select and modulate rays from the light field based on the needs of the application at hand.We have implemented a programmable imaging system that uses a digital micro-mirror device (DMD), which is used in digital light processing. Although the mirrors of this device can only be positioned in one of two states, we show that our system can be used to implement a wide variety of imaging functions, including, high dynamic range imaging, feature detection, and object recognition. We conclude with a discussion on how a micro-mirror array can be used to efficiently control field of view without the use of moving parts. A Flexible Approach to ImagingIn the past few decades, a wide variety of novel imaging systems have been proposed that have fundamentally changed the notion of a camera. These include high dynamic range, multispectral, omnidirectional, and multiviewpoint imaging systems. The hardware and software of each of these devices are designed to accomplish a particular imaging function and this function cannot be altered without significant redesign.In this paper, we introduce the notion of a programmable imaging system. Such a system gives a human user or a computer vision system significant control over the radiometric and geometric properties of the system. This flexibility is achieved by using a programmable array of micro-mirrors. The orientations of the mirrors of the array can be controlled with very high speed. This enables the system to select and modulate scene rays based on the needs of the application at hand. The end result is a single imaging system that can emulate the functionalities of several existing specialized systems as well as new ones. * This work was done at the Columbia Center for Vision and Graphics. It was supported by an ONR contract (N00014-03-1-0023). The basic principle behind the proposed approach is illustrated in Figure 1. The system observes the scene via a two-dimensional array of micro-mirrors, whose orientations can be controlled. The surface normal n i of the i th mirror determines the direction of the scene ray it reflects into the imaging system. If the normals of the mirrors can be arbitrarily chosen, each mirror can be programmed to select from a continuous cone of scene rays. In addition, each mirror can also be oriented with normal n b such that it reflects a black surface (with zero radiance). Let the integration time of the image detector be T . If the mirror is made to point in the directions n i and n b for durations t and T − t, respectively, the scene ray is attenuated by t/T . As a result, each imaged scene ray can also be radiometr...

Variable exposure time imaging for obtaining unblurred HDR images

“…Thus, these photographic characteristics were measured beforehand and were reflected to VET imaging in Eq. (8).…”

“…In order to obtain HDR images from a single image, new imaging methods were proposed recently [6][7][8]. In these methods, the image exposure is controlled pixel by pixel.…”

An overview of computational photography

“…Most typical computational optics try to improve the imaging results by controlling the aperture and capturing optically coded images, i.e., coded aperture imaging. In previous work, various finely designed modulators have been inserted into the aperture or added in front of the camera for the capture of optically coded images, including masks [3][4][5][6][7], pinhole patterns [8], lenslets [9][10][11], filters or occluders [12][13][14][15], and mirrors [16][17][18][19][20]; these elements are used in specific applications, such as capture of the light field [21], long distance camera interaction [22], defocus deblurring [23], extending the depth of field or refocusing [4][5][6][7][23][24][25], extending the field of view [26,27], confocal imaging [20], enhancing the dynamic range [13,18,28,29], extracting the depth cues [15], and capturing multi-modal visual information [30].…”

Temporal Dithering of Illumination for Fast Active Vision

“…Nayar et al [14] re-engineer a DLP projector into a DMD-camera and demonstrate the notion of programmable imaging for applications including adaptive dynamic range and optical filtering and matching. Based on the theory of compressive sampling, a single pixel camera has been implemented where the DMD device used to compute optical projections of scene radiance [15].…”