LiDAR/camera sensor fusion technology for pedestrian detection

Wu, Tung Hsin; Tsai, Chia-Chi; Guo, Jiun-In

doi:10.1109/apsipa.2017.8282301

Cited by 38 publications

(27 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiment, this was achieved by applying a modulo operator to the state of a 32-bit LFSR. The modulo was implemented as a bit-mask on the last W m LSBs of the LFSR, and the width of the bit-mask was selectable in W m ∈ [1,2,3,4,5,6]. The periodicity of the resulting sequence is determined by the length L of the LFSR, while the maximum applicable delay T d,max is defined by the bit-mask width W m and the system clock frequency F clk according to the relation Figure 12 reports the suppression ratio for a victim using a long LFSR (L = 32) as a function of the bit-mask width W m .…”

Section: Resultsmentioning

confidence: 99%

“…The use of LiDAR as a complement to existing technologies, such as radar, ultrasonic range finding, thermal imaging, and image processing, is rapidly changing the landscape of advanced automotive sensor systems. The joint use of heterogeneous sensors, known as sensor fusion, promises to facilitate the safe implementation of level 3 and level 4 autonomy [2][3][4], and to be an enabling factor of fully autonomous driving [5]. Even outside the automotive field, where the reliable and safe navigation of autonomous or semi-autonomous machines is needed (e.g., advanced robotics, automated guided vehicles, factory automation, simultaneous localization and mapping, and drones), LiDAR is proving itself to be a valid complement to existing navigation sensors [6,7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Optical Interference Suppression Scheme for TCSPC Flash LiDAR Imagers

Carrara¹,

Fiergolski²

2019

Applied Sciences

View full text Add to dashboard Cite

This paper describes an optical interference suppression scheme that allows flash light detection and ranging (LiDAR) imagers to run safely and reliably in uncontrolled environments where multiple LiDARs are expected to operate concurrently. The issue of optical interference is a potential show-stopper for the adoption of flash LiDAR as a technology of choice in multi-user application fields such as automotive sensing and autonomous vehicle navigation. The relatively large emission angle and field of view of flash LiDAR imagers make them especially vulnerable to optical interference. This work illustrates how a time-correlated single-photon counting LiDAR can control the timing of its laser emission to reduce its statistical correlation to other modulated or pulsed light sources. This method is based on a variable random delay applied to the laser pulse generated by LiDAR and to the internal circuitry measuring the time-of-flight. The statistical properties of the pseudorandom sequence of delays determines the effectiveness of LiDAR resilience against unintentional and intentional optical interference. For basic multi-camera operation, a linear feedback shift register (LFSR) was used as a random delay generator, and the performance of the interference suppression was evaluated as a function of sequence length and integration time. Direct interference from an identical LiDAR emitter pointed at the same object was reduced up to 50 dB. Changing integration time between 10 ms and 100 ms showed a marginal impact on the performance of the suppression (less than 3 dB deviation). LiDAR signal integrity was characterized during suppression, obtaining a maximum relative deviation of the measured time-of-flight of 0.1%, and a maximum deviation of measurements spread (full-width half-maximum) of 3%. The LiDAR signal presented an expected worst-case reduction in intensity of 25%.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Optical Interference Suppression Scheme for TCSPC Flash LiDAR Imagers

Carrara¹,

Fiergolski²

2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Feature fusion methods generally involve projecting the LiDAR point cloud into a 2D space and then processing both the image and the projection with some sort of CNN in order for the learned features that are extracted from both mediums to complement one another [27][28][29][30][31]. The other group of fusion algorithms is decision level fusions, which performs independent detections in both mediums and then combine both sets of detections together in order to output a single set of superior detections [32,33]. The idea behind fusion methods is to utilize the advantages of each type of data to augment one another in order to provide superior detection quality over each independent medium alone.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Fusion Object Detection System for Autonomous Vehicles

Person

Jensen

Smith

et al. 2019

Journal of Dynamic Systems, Measurement, and Control

View full text Add to dashboard Cite

In order for autonomous vehicles to safely navigate the road ways, accurate object detection must take place before safe path planning can occur. Currently, general purpose object detection convolutional neural network (CNN) models have the highest detection accuracies of any method. However, there is a gap in the proposed detection frameworks. Specifically, those that provide high detection accuracy necessary for deployment but do not perform inference in realtime, and those that perform inference in realtime but detection accuracy is low. We propose multimodel fusion detection system (MFDS), a sensor fusion system that combines the speed of a fast image detection CNN model along with the accuracy of light detection and range (LiDAR) point cloud data through a decision tree approach. The primary objective is to bridge the tradeoff between performance and accuracy. The motivation for MFDS is to reduce the computational complexity associated with using a CNN model to extract features from an image. To improve efficiency, MFDS extracts complimentary features from the LiDAR point cloud in order to obtain comparable detection accuracy. MFDS is novel by not only using the image detections to aid three-dimensional (3D) LiDAR detection but also using the LiDAR data to jointly bolster the image detections and provide 3D detections. MFDS achieves 3.7% higher accuracy than the base CNN detection model and is able to operate at 10 Hz. Additionally, the memory requirement for MFDS is small enough to fit on the Nvidia Tx1 when deployed on an embedded device.

show abstract

“…Development of sensor fusion techniques for automotive purposes has been on the rise in recent years. This is because these techniques have been able to provide a higher level of accuracy in detection [35]. Multispectral pedestrian and cyclist detection can be one of three categories: pixel-level (Early Fusion) (see Fig.…”

Section: Sensor Fusionmentioning

confidence: 99%

Visual and Thermal Data for Pedestrian and Cyclist Detection

Ahmed

Huda

Rajbhandari

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

With the continued advancement of autonomous vehicles and their implementation in public roads, accurate detection of vulnerable road users (VRUs) is vital for ensuring safety. To provide higher levels of safety for these VRUs, an effective detection system should be employed that can correctly identify VRUs in all types of environments (e.g. VRU appearance, crowded scenes) and conditions (e.g. fog, rain, night-time). This paper presents optimal methods of sensor fusion for pedestrian and cyclist detection using Deep Neural Networks (DNNs) for higher levels of feature abstraction. Typically, visible sensors have been utilised for this purpose. Recently, thermal sensors system or combination of visual and thermal sensors have been employed for pedestrian detection with advanced detection algorithm. DNNs have provided promising results for improving the accuracy of pedestrian and cyclist detection. This is because they are able to extract features at higher levels than typical hand-crafted detectors. Previous studies have shown that amongst the several sensor fusion techniques that exist, Halfway Fusion has provided the best results in terms of accuracy and robustness. Although sensor fusion and DNN implementation have been used for pedestrian detection, there is considerably less research undertaken for cyclist detection.

show abstract

LiDAR/camera sensor fusion technology for pedestrian detection

Cited by 38 publications

References 3 publications

An Optical Interference Suppression Scheme for TCSPC Flash LiDAR Imagers

An Optical Interference Suppression Scheme for TCSPC Flash LiDAR Imagers

Multimodal Fusion Object Detection System for Autonomous Vehicles

Visual and Thermal Data for Pedestrian and Cyclist Detection

Contact Info

Product

Resources

About