Vehicle detection based on LiDAR and camera fusion

Zhang, Feihu; Clarke, Daniel; Knoll, Alois

doi:10.1109/itsc.2014.6957925

Cited by 69 publications

(42 citation statements)

References 26 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea behind fusion methods is to utilize the advantages of each type of data to augment one another in order to provide superior detection quality over each independent medium alone. The advantages of fusion methods are not only relegated to learned features either and can benefit handcrafted feature extraction methods [34,35]. A novel approach that was developed at the same time as the proposed algorithm is FrustrumNet [36].…”

Section: Related Workmentioning

confidence: 99%

Multimodal Fusion Object Detection System for Autonomous Vehicles

Person

Jensen

Smith

et al. 2019

Journal of Dynamic Systems, Measurement, and Control

View full text Add to dashboard Cite

In order for autonomous vehicles to safely navigate the road ways, accurate object detection must take place before safe path planning can occur. Currently, general purpose object detection convolutional neural network (CNN) models have the highest detection accuracies of any method. However, there is a gap in the proposed detection frameworks. Specifically, those that provide high detection accuracy necessary for deployment but do not perform inference in realtime, and those that perform inference in realtime but detection accuracy is low. We propose multimodel fusion detection system (MFDS), a sensor fusion system that combines the speed of a fast image detection CNN model along with the accuracy of light detection and range (LiDAR) point cloud data through a decision tree approach. The primary objective is to bridge the tradeoff between performance and accuracy. The motivation for MFDS is to reduce the computational complexity associated with using a CNN model to extract features from an image. To improve efficiency, MFDS extracts complimentary features from the LiDAR point cloud in order to obtain comparable detection accuracy. MFDS is novel by not only using the image detections to aid three-dimensional (3D) LiDAR detection but also using the LiDAR data to jointly bolster the image detections and provide 3D detections. MFDS achieves 3.7% higher accuracy than the base CNN detection model and is able to operate at 10 Hz. Additionally, the memory requirement for MFDS is small enough to fit on the Nvidia Tx1 when deployed on an embedded device.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal Fusion Object Detection System for Autonomous Vehicles

Person

Jensen

Smith

et al. 2019

Journal of Dynamic Systems, Measurement, and Control

View full text Add to dashboard Cite

show abstract

“…The hypothesis verification phase is implemented with the same procedure in our previous work [19] which also consists of two steps: the parameter estimation and object classification. Shape parameters is calculated by the Random Hypersurface Model while the classification is implemented with the Support Vector Machine.…”

Section: Hypothesis Verification Phasementioning

confidence: 99%

“…Example for the shape of star-convex object [19] where the state, the measurement noise and the measurement are mapped to a non-linear condition. Finally, the Unscented Kalman filter (UKF) is utilized to estimate the corresponding states.…”

Section: A Parameter Estimationmentioning

confidence: 99%

See 1 more Smart Citation

LiDAR based vehicle detection in urban environment

Zhang

Clarke

Knoll

2014

2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI)

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, a LiDAR based vehicle detection approach is proposed with the goal of utilizing range information. The proposed approach is based on two phases: a hypothesis generation phase to generate the potential regions and a hypothesis verification phase to recognize the corresponding vehicles. In contrast to appearance based vehicle detection systems, the proposed approach solely relies on the range information and achieves a close performance to the state-of-art. Furthermore, the proposed approach is adaptable to the environment constrains in contrast to vision based techniques, e.g. light intensity and fields of view.Performance of the proposed approach is evaluated on a large public dataset in urban environment.

show abstract

“…[12]. Combination of RGB image and point cloud data has also been studied in some articles [18] [19].…”

Section: List Of Tablesmentioning

confidence: 99%

Multi-Sensor Data Fusion for Robust Environment Reconstruction in Autonomous Vehicle Applications

Rubaiyat¹

View full text Add to dashboard Cite

In autonomous vehicle systems, understanding the surrounding environment is mandatory for an intelligent vehicle to make every decision of movement on the road. Knowledge about the neighboring environment enables the vehicle to detect moving objects, especially irregular events such as jaywalking, sudden lane change of the vehicle etc. to avoid collision. This local situation awareness mostly depends on the advanced sensors (e.g. camera, LIDAR, RADAR) added to the vehicle. The main focus of this work is to formulate a problem of reconstructing the vehicle environment using point cloud data from the LIDAR and RGB color images from the camera. Based on a widely used point cloud registration tool such as iterated closest point (ICP), an expectationmaximization (EM)-ICP technique has been proposed to automatically mosaic multiple point cloud sets into a larger one. Motion trajectories of the moving objects are analyzed to address the issue of irregularity detection. Another contribution of this work is the utilization of fusion of color information (from RGB color images captured by the camera) with the three-dimensional point cloud data for better representation of the environment. For better understanding of the surrounding environment, histogram of oriented gradient (HOG) based techniques are exploited to detect pedestrians and vehicles. Using both camera and LIDAR, an autonomous vehicle can gather information and reconstruct the map of the surrounding environment up to a certain distance. Capability of communicating and cooperating among vehicles can improve the automated driving decisions by providing extended and more precise view of the surroundings. In this work, a transmission power control algorithm is studied along with the adaptive content control algorithm to achieve a more accurate map of the vehicle environment. To exchange the local sensor data among the vehicles, an adaptive communication scheme is proposed that controls the lengths and the contents of the messages depending on the load of the communication channel. The exchange of this information can extend the tracking region of a vehicle beyond the area sensed by its own sensors. In this experiment, a combined effect of power control, and message length and content control algorithm is exploited to improve the map's accuracy of the surroundings in a cooperative automated vehicle system. I would also like to convey my heartfelt thanks to Dr. Xin Li for his continuous guidance. His advice and innovative ideas helped me to successfully complete my thesis. I greatly appreciate my thesis committee members, Dr. Parviz Famouri and Dr. Muhammad A. Choudhry, for their time and support. I wish to acknowledge the Lane Department of Computer Science and Electrical Engineering to give me the opportunity of pursuing my master's degree in such an academically vibrant environment. I would like to thank my family for supporting me throughout my studies. I am also thankful to the members of my research group for their help during my research at WVU.

show abstract

Vehicle detection based on LiDAR and camera fusion

Cited by 69 publications

References 26 publications

Multimodal Fusion Object Detection System for Autonomous Vehicles

Multimodal Fusion Object Detection System for Autonomous Vehicles

LiDAR based vehicle detection in urban environment

Multi-Sensor Data Fusion for Robust Environment Reconstruction in Autonomous Vehicle Applications

Contact Info

Product

Resources

About