High-speed 3D sensing via hybrid-mode imaging and guided upsampling

Abstract

Imaging systems with temporal resolution play a vital role in a diverse range of scientific, industrial, and consumer applications, e.g., fluorescent lifetime imaging in microscopy and time-of-flight (ToF) depth sensing in autonomous vehicles. In recent years, single-photon avalanche diode (SPAD) arrays with picosecond timing capabilities have emerged as a key technology driving these systems forward. Here we report a high-speed 3D imaging system enabled by a state-of-the-art SPAD sensor used in a hybrid imaging mode that can perform multi-event histogramming. The hybrid imaging modality alternates between photon counting and timing frames at rates exceeding 1000 frames per second, enabling guided upscaling of depth data from a native resolution of 64 × 32 to 256 × 128 . The combination of hardware and processing allows us to demonstrate high-speed ToF 3D imaging in outdoor conditions and with low latency. The results indicate potential in a range of applications where real-time, high throughput data are necessary. One such example is improving the accuracy and speed of situational awareness in autonomous systems and robotics.

SPADs and SiPMs Arrays for Long-Range High-Speed Light Detection and Ranging (LiDAR)

“…The 2019 Hutchings [ 46 ] SPAD array can work either in photon counting (with full 256 × 256 resolution) or in photon timing mode, connecting groups of 4 × 4 SPADs to the same processing unit (with reduced 64 × 64 resolution). The detector is able to fast switch between the two modalities, so to perform hybrid intensity high-resolution images and 3D maps [ 69 ]. In timing mode, the detector can either work in single-hit high temporal resolution mode (38 ps resolution) or in multi-event histogramming mode (560 ps resolution).…”

“…In Hutchings’ study [ 46 ], the image resolution has been improved by combining 2D high resolution imaging (each of the 256 × 256 SPADs works independently) with 3D lower resolution maps (SPADs are combined in macropixels, with 4 × 4 detectors each). Since 2D and 3D information cannot be simultaneously acquired (because some hardware resources are shared by the two modalities), the detector is used in a hybrid mode with interleaved 2D and 3D frames, as shown in [ 69 ]. Data-fusion proposed in [ 70 ], further allows to improve image resolution, by combining information from a time-resolved 240 × 320 pixels SPAD array with a co-registered high-resolution red-green-blue (RGB) digital camera.…”

A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar

“…In general, these approaches only register the first photon in the frame therefore multiple laser cycles are required to build an accurate timing statistic before traversing to a new point in the scene. In high ambient conditions, a significant pile-up effect can occur due to the dead-time of the single SPAD [26]. This problem can be alleviated by using multiple SPADs in parallel resulting in multi-event time-stamp collection [26].…”

A novel contactless technique to measure water waves using a single photon avalanche diode detector array

“…Traditionally realized as single-point detectors (requiring optical scanning to cover a field of view), with separate timing modules, recent developments have resulted in array format SPAD sensors with integrated photon timing and processing electronics [23]. These advances are now supporting advanced ToF imaging, including underwater [24] and at high speeds [25].…”