Feature extraction and neural network-based multi-peak analysis on time-correlated LiDAR histograms

Abstract

Time correlated single photon counting (TCSPC) is a statistical method to generate time-correlated histograms (TC-Hists), which are based on the time-of-flight (TOF) information measured by photon detectors such as single-photon avalanche diodes. With restricted measurements per histogram and the presence of high background light, it is challenging to obtain the target distance in a TC-Hist. In order to improve the data processing robustness under these conditions, the concept of machine learning is applied to the TC-Hist. Using the neural network-based multi-peak analysis (NNMPA), introduced by us, including a physics-guided feature extraction, a neural network multi-classifier, and a distance recovery process, the analysis is focused on a small amount of critical features in the TC-Hist. Based on these features, possible target distances with correlated certainty values are inferred. Furthermore, two optimization approaches regarding the learning ability and real-time performance are discussed. In particular, variants of the NNMPA are evaluated on both synthetic and real datasets. The proposed method not only has higher robustness in allocating the coarse position (±5 %) of the target distance in harsh conditions, but also is faster than the classical digital processing with an average-filter. Thus, it can be applied to improve the system robustness, especially in the case of high background light and middle-range detections.

MSNR: A Multistage Noise Reduction Method on Point Cloud+

“…MSNR uses the distance prediction results of NNMPA [5] as input. Specifically, the data pairs {(𝑑 1 , 𝑠 1 ), (𝑑 2 , 𝑠 2 ), ... } of each pixel are used, where 𝑑 𝑛 refers to the 𝑛 th potential distance and 𝑠 𝑛 refers to the probability value to the 𝑛 th distance (higher 𝑠 𝑛 indicate that the corresponding distance is more likely to be the true distance).…”

“…1) Border-Effect Suppression: As discussed in [5], NNMPA suffers from the border effect, when the object locates at the junction of the two sub-regions. In this case, the distance information of an object is captured twice, resulting in two neighbouring distance points.…”

“…2) Conditional Otsu Filtering: According to [5], 𝑠 𝑛 refers to the probability of the distance being the object distance. Thus, the conditional Otsu filtering, modified from the automatical thresholding method Otsu [15], is applied to remove the noise points with small 𝑠 𝑛 in three steps:…”

“…A SPAD-based d-ToF LiDAR system has a significant amount of noise in the raw data under harsh measurement conditions, e. g. high background light. Neural network-based multi-peak-analysis (NNMPA) [5] is proposed as a software solution to cope with this problem. It extracts important features from LiDAR histograms and determining distance information with a neural network.…”

“…In this paper, a point cloud filtering method with multi-stage noise reduction (MSNR) is proposed. MSNR takes the output generated by NNMPA [5] as input and filters noise points out. The work is structured as follows: In section II, existing methods for improving the point cloud are introduced.…”