Practical classification of different moving targets using automotive radar and deep neural networks

Abstract

In this work, we present results for classification of different classes of targets (car, single and multiple people, bicycle) using automotive radar data and different neural networks. A fast implementation of radar algorithms for detection, tracking, and micro-Doppler extraction is proposed in conjunction with the automotive radar transceiver TEF810X and microcontroller unit SR32R274 manufactured by NXP Semiconductors. Three different types of neural networks are considered, namely a classic convolutional network, a residual network, and a combination of convolutional and recurrent network, for different classification problems across the 4 classes of targets recorded. Considerable accuracy (close to 100% in some cases) and low latency of the radar pre-processing prior to classification (approximately 0.55s to produce a 0.5s long spectrogram) are demonstrated in this paper, and possible shortcomings and outstanding issues are discussed.

CNN Based Road User Detection Using the 3D Radar Cube

“…Doppler-time features were also used in automotive setups. [6] applies a CNN-LSTM network on Range-Doppler and Doppler-Time spectrograms of 0.5-2 seconds to classify pedestrian, group of pedestrians, car, and cyclist classes. [10] pointed out that a long multi-frame observation period is not viable for urban driving, and proposed a single-frame usage of low-level data.…”

“…Digital Object Identifier (DOI): see top of this page. Various methods [4], [5], [6] instead explore using the lowlevel radar cube extracted from an earlier signal processing stage of the radar. The radar cube is a 3D data matrix with axes corresponding to range, azimuth, and velocity (also called Doppler), and a cell's value represents the measured radar reflectivity in that range/azimuth/Doppler bin.…”

“…Radar targets are shown as dots, colored green/blue for pedestrian/car ground truth class. radar cube features are computed by first generating 2D rangeazimuth or range-Doppler projections, or by aggregating the projected Doppler axis over time into a Doppler-time image [6], [7]. We will call features derived from the 3D cube or its projections low-level.…”

Experiments with mmWave Automotive Radar Test-bed

“…[5] used different deep learning methods to extract micro-Doppler patterns in the STFT heatmap, with up to 93% recognition accuracy when evaluated for three class discrimination: car, pedestrian and cyclist. However, the data of [5] was obtained solely from single input, single output (SISO) radar where only one range bin was used to generate STFT heatmap. Compared to MIMO radar, SISO contains little information about the shape of extended objects, that can greatly improve object recognition and classification.…”

“…Compared to MIMO radar, SISO contains little information about the shape of extended objects, that can greatly improve object recognition and classification. Also, the evaluation of [5] didn't take the detection error into account and hence the accuracy metric would not demonstrate its ability to deal with missing detection and false alarm.…”