CNN Based Road User Detection Using the 3D Radar Cube

Palffy, Andras; Dong, Jiaao; Kooij, Julian F. P.; Gavrila, Dariu M.

doi:10.1109/lra.2020.2967272

Cited by 133 publications

(86 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our cooperative fusion algorithm requires a complete range-Doppler-azimuth radar cube to perform accurate, high recall, radar detection making available datasets inadequate. Lastly, the dataset most applicable to our work was captured by the Intelligent Vehicles at TU Delft [ 55 ] and offers synchronized camera and raw radar 3D cubes. Unfortunately, due to a non-disclosure agreement issues the complete data is not available at the time of writing and simulations are provided as placeholders.…”

Section: Evaluation and Resultsmentioning

confidence: 99%

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Dimitrievski

Hamme

Veelaert

et al. 2020

Sensors

View full text Add to dashboard Cite

This paper presents a vulnerable road user (VRU) tracking algorithm capable of handling noisy and missing detections from heterogeneous sensors. We propose a cooperative fusion algorithm for matching and reinforcing of radar and camera detections using their proximity and positional uncertainty. The belief in the existence and position of objects is then maximized by temporal integration of fused detections by a multi-object tracker. By switching between observation models, the tracker adapts to the detection noise characteristics making it robust to individual sensor failures. The main novelty of this paper is an improved imputation sampling function for updating the state when detections are missing. The proposed function uses a likelihood without association that is conditioned on the sensor information instead of the sensor model. The benefits of the proposed solution are two-fold: firstly, particle updates become computationally tractable and secondly, the problem of imputing samples from a state which is predicted without an associated detection is bypassed. Experimental evaluation shows a significant improvement in both detection and tracking performance over multiple control algorithms. In low light situations, the cooperative fusion outperforms intermediate fusion by as much as 30%, while increases in tracking performance are most significant in complex traffic scenes.

show abstract

Section: Evaluation and Resultsmentioning

confidence: 99%

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Dimitrievski

Hamme

Veelaert

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The Range-Doppler-Azimuth spectrums extracted from automotive radar sensors have been used frequently as 2D image inputs to various deep learning algorithms for different tasks, ranging from obstacle detection to segmentation, classification, and identification in autonomous driving systems [ 122 , 123 , 124 , 125 , 126 ]. The authors of [ 122 ] presented a method to recognize objects in Cartesian coordinates using a high-resolution 300-GHz scanning radar based on deep neural networks.…”

Section: Detection and Classification Of Radar Signals Using Deepmentioning

confidence: 99%

Application of Deep Learning on Millimeter-Wave Radar Signals: A Review

Abdu

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

The progress brought by the deep learning technology over the last decade has inspired many research domains, such as radar signal processing, speech and audio recognition, etc., to apply it to their respective problems. Most of the prominent deep learning models exploit data representations acquired with either Lidar or camera sensors, leaving automotive radars rarely used. This is despite the vital potential of radars in adverse weather conditions, as well as their ability to simultaneously measure an object’s range and radial velocity seamlessly. As radar signals have not been exploited very much so far, there is a lack of available benchmark data. However, recently, there has been a lot of interest in applying radar data as input to various deep learning algorithms, as more datasets are being provided. To this end, this paper presents a survey of various deep learning approaches processing radar signals to accomplish some significant tasks in an autonomous driving application, such as detection and classification. We have itemized the review based on different radar signal representations, as it is one of the critical aspects while using radar data with deep learning models. Furthermore, we give an extensive review of the recent deep learning-based multi-sensor fusion models exploiting radar signals and camera images for object detection tasks. We then provide a summary of the available datasets containing radar data. Finally, we discuss the gaps and important innovations in the reviewed papers and highlight some possible future research prospects.

show abstract

“…As these signals are given in the form of a vector with a single time index, they must be converted into a matrix form of size N × M to check the range-Doppler responses, as shown in (6). The N M data samples sampled in the ADC are converted to the matrix form of (6), and then the range-Doppler response is obtained through the processes of (8)- (11). Analyzing the magnitude of range-Doppler responses, as presented in Fig.…”

Section: B Range-doppler Response Of the Interference Signalmentioning

confidence: 99%

Classification of Interference Signal for Automotive Radar Systems With Convolutional Neural Network

et al. 2020

View full text Add to dashboard Cite

When a radar signal generated by another vehicle arrives at an ego-vehicle, mutual interference occurs, which can seriously degrade the detection performance of the radar. To mitigate mutual interference, the type of radar modulation used in the interference vehicle must be identified because the types of radar systems installed in each vehicle are different. Therefore, in this paper, we propose a method for classifying the modulation types of interference signals in automotive fast chirp frequency modulated continuous waveform (FMCW) radar systems. We build a mathematical model of the received signal when the radar signal transmitted by the ego-vehicle interferes with various types of interference signals, such as unmodulated continuous wave (CW), slow chirp FMCW, fast chirp FMCW, pulsed CW, and frequency-shift keying signals. In the fast chirp FMCW radar systems, the received signal is converted into range-Doppler response using two-dimensional Fourier transform. Based on range-Doppler responses of the interference signals, we design a classifier to identify the modulation type of interference signals using a convolutional neural network (CNN). Through our proposed CNN, we can classify five different types of interference signals with an accuracy of over 96%. In addition, compared to conventional feature-based machine learning techniques such as support vector machines, the proposed method can effectively identify the interference signal with fewer input signals in shorter time. INDEX TERMS Automotive radar, modulation type classification, mutual interference, convolutional neural network (CNN).

show abstract

CNN Based Road User Detection Using the 3D Radar Cube

Cited by 133 publications

References 26 publications

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Application of Deep Learning on Millimeter-Wave Radar Signals: A Review

Classification of Interference Signal for Automotive Radar Systems With Convolutional Neural Network

Contact Info

Product

Resources

About