Doppler time-of-flight imaging

Abstract

Figure 1: We introduce a new computational imaging system that allows for metric radial velocity information to be captured instantaneously for each pixel (center row). For this purpose, we design the temporal illumination and modulation frequencies of a time-of-flight camera (left) to be orthogonal within its exposure time. The Doppler effect of objects in motion is then detected as a frequency shift of the illumination, which results in a mapping from object velocity to recorded pixel intensity. By capturing a few coded time-of-flight measurements and adding a conventional RGB camera to the setup, we demonstrate that color, velocity, and depth information of a scene can be recorded simultaneously. The results above show several frames of two video sequences. For each example, the left-most frame shows a static object (velocity map is constant), which is then moved towards (positive radial velocity) or away from (negative velocity) the camera. AbstractOver the last few years, depth cameras have become increasingly popular for a range of applications, including human-computer interaction and gaming, augmented reality, machine vision, and medical imaging. Many of the commercially-available devices use the time-of-flight principle, where active illumination is temporally coded and analyzed in the camera to estimate a per-pixel depth map of the scene. In this paper, we propose a fundamentally new imaging modality for all time-of-flight (ToF) cameras: per-pixel radial velocity measurement. The proposed technique exploits the Doppler effect of objects in motion, which shifts the temporal illumination frequency before it reaches the camera. Using carefully coded illumination and modulation frequencies of the ToF camera, object velocities directly map to measured pixel intensities. We show that a slight modification of our imaging system allows for color, depth, and velocity information to be captured simultaneously. Combining the optical flow computed on the RGB frames with the measured metric radial velocity allows us to further estimate the full 3D metric velocity field of the scene. The proposed technique has applications in many computer graphics and vision problems, for example motion tracking, segmentation, recognition, and motion deblurring.

Methods for linear radial motion estimation in time-of-flight range imaging

“…DoP by six phase steps is possible (three per phase measurement), but at an increase in noise. 3 6 ≤ 0.02 NR Whyte et al 2 1.1 0.003-0.034 NR (R 2 = 0.9969)…”

“…But, motion can also be treated as another variable in ToF imaging, especially radial motion which manifests as a Doppler shift. 2,3 This work is concerned with the measurement of linear radial motion (linear velocity towards or away from the camera aperture), and mitigation of error due to radial motion in measurements of distance.…”

Gated2Depth: Real-Time Dense Lidar From Gated Images

“…However, in addition to the modulated light, this sensing approach also records all ambient background light. While per-pixel lock-in amplification removes background components efficiently in indoor scenarios [33], and learned architectures can alleviate multi-path distortions [55], existing C-ToF cameras are limited to ranges of a few meters in outdoor scenarios [22] in strong sunlight. Gated cameras send out pulses of flood-light and only record photons from a certain distance by opening and closing the camera after a given delay.…”

Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset

“…After detection, pixels affected by motion blur can be corrected, for example, by interpolation [14]. Object motion also affects the frequency of reflected signals due to Doppler effect; however, to use the frequency shift to measure object motion requires a higher number of measurements and extensive processing [16].…”

“…An AMCW ToF camera illuminates the scene with an amplitude-modulated, sinusoidal signal g(t) = g 1 cos(ωt) + g 0 , [16]. If camera and light source are colocated, and the scene static, the signal coming back to a pixel is…”