A calibration scheme for non-line-of-sight imaging setups

Abstract

The recent years have given rise to a large number of techniques for “looking around corners”, i.e., for reconstructing or tracking occluded objects from indirect light reflections off a wall. While the direct view of cameras is routinely calibrated in computer vision applications, the calibration of non-line-of-sight setups has so far relied on manual measurement of the most important dimensions (device positions, wall position and orientation, etc.). In this paper, we propose a method for calibrating time-of-flight-based non-line-of-sight imaging systems that relies on mirrors as known targets. A roughly determined initialization is refined in order to optimize for spatio-temporal consistency. Our system is general enough to be applicable to a variety of sensing scenarios ranging from single sources/detectors via scanning arrangements to large-scale arrays. It is robust towards bad initialization and the achieved accuracy is proportional to the depth resolution of the camera system.

Onsite Non-Line-of-Sight Imaging via Online Calibrations

“…In NLOS imaging, positions and distances of the hardware devices need to be under control to record precise transients of hidden objects. Klein et al [23] proposed a calibration scheme with mirrors as target objects placed at different positions in an NLOS scene. Since the laser is directed by a 2D galvanometer, digital cameras are useful tools to take pictures of laser spots on the relay surface and to determine the 3D coordinates of the spots using computer vision techniques [4,6,22].…”

“…For example, a checkerboard or regular point grid coupled with a visible camera [4,22] can be used to acquire the internal and external parameters of the camera or the 3D coordinates of the grid point. Alternatives include using mirrors [23] to establish correspondences of the laser point and the detection point for simultaneously estimating the mirror plane and the relay wall. When the scanning point is re-positioned, or the position of the relay surface changes relative to the data acquisition system, these methods need re-calibration.…”

Onsite Non-Line-of-Sight Imaging via Online Calibration

“…Buttafava et al [4] and Ahn et al [18] (Ahn [18]) exploit a digital camera and a checkerboard (or a regular grid) to estimate camera parameters, the relay surface, and 3D coordinates of detection points based on computer vision (CV) techniques. Klein et al [19] use mirrors to establish correspondences of the laser spots and the detection points for simultaneously estimating the mirror plane and the relay surface. When readjusting the configuration of the imaging system, these methods repeat the whole process of calibration, which is both labor-intensive and time-consuming for onsite deployment.…”

“…Table I lists important parameters of several single-pixel SPAD-based NLOS imaging systems in the literature. Two major types of SPADs are affordable in terms of material properties: silicon-based SPADs cover the visible spectrum, e.g., 400-800 nm, with full width at half maximum (FWHM) of tens of ps [5]- [7], whereas InGaAs/InP-based SPADs cover the infrared spectrum with FWHM of approximately 200 ps [10], [11], [19], [23]. Wu et al [10] exploit InGaAs/InP SPADs to construct a long-range NLOS imaging system, over 1.43 km.…”

“…Calibration schemes for NLOS imaging systems. Klein et al [19] employ a 32×32 InGaAs/InP SPAD array to construct an NLOS imaging system under non-confocal setting. Analogous to the classical calibration of a digital camera in the CV community, they propose a calibration scheme with mirrors as known targets placed at different positions in an NLOS scene.…”