Detection and tracking of moving objects hidden from view

Abstract

The ability to detect motion and track a moving object hidden around a corner or behind a wall provides a crucial advantage when physically going around the obstacle is impossible or dangerous. Previous methods have demonstrated that is possible to reconstruct the shape of an object hidden from view. However, these methods do not enable the tracking of movement in real-time. We demonstrate a compact non-line-of-sight laser ranging technology that relies upon the ability to send light around an obstacle using a scattering floor and to detect the return signal from a hidden object with only a few seconds acquisition time. By detecting this signal with a single-photon avalanche diode (SPAD) camera, we follow the movement of an object located a meter away from the camera with centimetre precision. We discuss the possibility of applying this technology to a variety of real-life situations in the 1 near future.Recent years have seen remarkable advances in the field of image processing and data acquisition, allowing for a range of novel applications [1][2][3][4][5][6][7][8]. An exciting new avenue is using optical imaging techniques to observe and track objects that are both in movement and hidden from the direct line-of-sight. The ability to detect motion and track a moving object hidden from view would provide a crucial advantage when physically going around the obstacle is impossible or dangerous, for example to detect a person moving behind a wall or a car approaching from behind a blind corner.Techniques for imaging static objects that are hidden from view have been recently demonstrated relying on, for example, radar technology [9,10], variations of laser illuminated detection and ranging (LIDAR) [3,5,11,12], or speckle-based imaging. The latter approach was first developed for imaging through opaque barriers [13][14][15], and also allows for imaging around corners [16,17]. The work of Velten et al. [5] and, more recently, Buttafava et al. [8] sets out to establish the 3D shape of a static hidden object by collecting the return scattered light with a streak camera or single-photon avalanche diode, respectively. While remarkable 3D reconstruction of objects are achieved with these techniques, Buttafava et al. point out that the requirement for scanning and subsequent long acquisition times mean that their technique is currently unsuitable for imaging moving objects.Notwithstanding these ingenious imaging systems, locating the position of a hidden object in motion and monitoring its movement in real time remains to date a major challenge. We set out to solve the tracking problem and develop a technique based on both hardware and software implementations that are specifically designed for this 2 purpose. Our solution is based on a LIDAR-like approach where a single-photon avalanche diode (SPAD) camera [7,[18][19][20][21][22] is used to image light that is backscattered from beyond the direct line-of-sight (see Methods for camera details). The high temporal resolution of the camera relies on the fact that each indiv...

Picosecond time-resolved imaging using SPAD cameras

“…The algorithm used to retrieve the objects position, as well as a proof-of-principle experiment at small scale (30 cm high object located 1 m away from the camera), have been reported in. 7 To retrieve the position of a hidden object we first measure the time of arrival of the signal of interest of each of the 32x32 pixels of the SPAD array. For a pixel imaging a spot on the floor at position r i , the time of flight measured at this pixel corresponds to the time of flight of the laser pulses between the laser spot on the floor ( r l ) and the object ( r o ), and between the object and the pixel i.…”

“…We therefore integrated the signal over 30 seconds. The signal from the target is isolated in a similar way than that reported in, 7 where a background is subtracted to the acquired data to isolate the signal of interest from all other signals coming from the environment. A Gaussian is fitted to each detected histogram to find the signal arrival time at each pixel, and this information is used to retrieve the hidden object's position.…”

Tracking objects outside the line of sight using 2D intensity images

“…A particularly fascinating result is the use of ultrafast time-of-flight measurements12 to image objects outside the direct line of sight3456. Being able to use arbitrary walls as though they were mirrors can provide a critical advantage in many sensing scenarios with limited visibility, like endoscopic imaging, automotive safety, industrial inspection and search-and-rescue operations.…”

“…We adopt the significantly more challenging assumption that the object is in the direct line of sight of neither light source nor camera (Fig. 1), and that it can only be illuminated or observed indirectly via a diffuse wall345614. All the observed light has undergone at least three diffuse reflections (wall, object, wall), and reconstructing the unknown object is an ill-posed inverse problem.…”

Observation of laser pulse propagation in optical fibers with a SPAD camera

“…To alleviate this, the outgoing beam can be scanned to form an image , sub-Rayleigh imaging 17 , and for ranging and 3D reconstruction with an external time to digital converter (TDC) 18 . These techniques have been extended to applications such as detecting objects hidden from view 19,20 and tracking moving objects around a corner 21 . Recently, light-in-flight imaging was implemented to visualize a laser pulse propagating in air 22 .…”