Multi-Class Road User Detection With 3+1D Radar in the View-of-Delft Dataset

Abstract

Next-generation automotive radars provide elevation data in addition to range-, azimuth-and Doppler velocity. In this experimental study, we apply a state-of-the-art object detector (PointPillars), previously used for LiDAR 3D data, to such 3+1D radar data (where 1D refers to Doppler). In ablation studies, we first explore the benefits of the additional elevation information, together with that of Doppler, radar cross section and temporal accumulation, in the context of multi-class road user detection. We subsequently compare object detection performance on the radar and LiDAR point clouds, object class-wise and as a function of distance. To facilitate our experimental study, we present the novel View-of-Delft (VoD) automotive dataset. It contains 8693 frames of synchronized and calibrated 64-layer LiDAR-, (stereo) camera-, and 3+1D radar-data acquired in complex, urban traffic. It consists of 123106 3D bounding box annotations of both moving and static objects, including 26587 pedestrian, 10800 cyclist and 26949 car labels. Our results show that object detection on 64-layer LiDAR data still outperforms that on 3+1D radar data, but the addition of elevation information and integration of successive radar scans helps close the gap. The VoD dataset is made freely available for scientific benchmarking at https://intelligent-vehicles.org/datasets/view-ofdelft/.

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

“…The class distribution is very imbalanced, with over 90% of the annotated objects being car. The View-of-Delft (VoD) dataset [50] is a recently published 4D radar dataset especially focused on the detection of VRUs. The sensor suite includes a ZF FRGen 21 4D radar, a 64-beam LiDAR, and a stereo camera.…”

“…As illustrated in Figure 6, rotating objects locally will affect the radial velocity, and rotating the radar point cloud globally will affect the ego-motion-compensated radial velocity if the ego-motion is not rotated accordingly. Therefore, Palffy et al [50] advise only using mirroring and scaling along the longitudinal axis as augmentation. Another applicable technique is the copy-paste augmentation, which copies the detected object from other frames and pastes it into the same location in the current frame, as done in [89].…”

“…Long et al [91] compensated the accumulated radar point cloud with the estimated full velocity, achieving better performance in bounding box regression. Plaffy et al [50] augmented the accumulated radar point cloud by appending a temporal index to each point as an additional channel. Along with the increased density, this index augmentation is expected to effectively retain the temporal information.…”

“…Therefore, it is possible to train a single detector for both static and dynamic objects. Plaffy et al [50] applied PointPillars [152] to 4D radar point clouds for 3D detection of multi-class road users. They found the performance can be improved by temporal integration and by introducing additional features, such as elevation, Doppler velocity, and RCS.…”

“…However, the performance evaluated on research datasets could be biased due to class imbalance and simple scenarios. Palffy et al [50] summarised some failure cases for radar detection in the VoD dataset: Two close pedestrians can be detected as one bicyclist. One large object, for example a truck or a bus, can be split into two smaller ones.…”