Occlusion aware sensor fusion for early crossing pedestrian detection

Palffy, Andras; Kooij, Julian F. P.; Gavrila, Dariu M.

doi:10.1109/ivs.2019.8814065

Cited by 24 publications

(18 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that even VRUs in occlusion (see Fig. 5a, 5b, 5g) are often classified correctly caused by the multi-path propagation of radar [8]. This, and its uniform performance in darkness/shadows/bright environments makes radar a useful complementary sensor for camera.…”

Section: Discussionmentioning

confidence: 95%

“…Target-level class labels are valuable for sensor fusion operating on intermediate-level, i.e. handling multiple measurements per object [8], [9]. Our targetlevel classification is more robust than cluster-wise classification where the initial clustering step must manage to separate radar targets from different objects, and keep those coming from the same object together, see Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CNN Based Road User Detection Using the 3D Radar Cube

Palffy

Dong

Kooij

et al. 2020

IEEE Robot. Autom. Lett.

Self Cite

133

View full text Add to dashboard Cite

This paper presents a novel radar based, singleframe, multi-class detection method for moving road users (pedestrian, cyclist, car), which utilizes low-level radar cube data. The method provides class information both on the radar targetand object-level. Radar targets are classified individually after extending the target features with a cropped block of the 3D radar cube around their positions, thereby capturing the motion of moving parts in the local velocity distribution. A Convolutional Neural Network (CNN) is proposed for this classification step. Afterwards, object proposals are generated with a clustering step, which not only considers the radar targets' positions and velocities, but their calculated class scores as well. In experiments on a real-life dataset we demonstrate that our method outperforms the state-of-the-art methods both target-and objectwise by reaching an average of 0.70 (baseline: 0.68) target-wise and 0.56 (baseline: 0.48) object-wise F1 score. Furthermore, we examine the importance of the used features in an ablation study.

show abstract

Section: Discussionmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

CNN Based Road User Detection Using the 3D Radar Cube

Palffy

Dong

Kooij

et al. 2020

IEEE Robot. Autom. Lett.

Self Cite

133

View full text Add to dashboard Cite

show abstract

“…In the past years, many sensor fusion methods have been proposed for autonomous driving applications [13][14][15][16]. In addition, [17][18][19][20][21] examine the existing problems of the current sensor fusion algorithm. According to different data processing methods, sensor fusion can be divided into three levels: the data layer, feature layer and decision layer.…”

Section: Introductionmentioning

confidence: 99%

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

et al. 2021

View full text Add to dashboard Cite

The development of autonomous vehicles and unmanned aerial vehicles has led to a current research focus on improving the environmental perception of automation equipment. The unmanned platform detects its surroundings and then makes a decision based on environmental information. The major challenge of environmental perception is to detect and classify objects precisely; thus, it is necessary to perform fusion of different heterogeneous data to achieve complementary advantages. In this paper, a robust object detection and classification algorithm based on millimeter-wave (MMW) radar and camera fusion is proposed. The corresponding regions of interest (ROIs) are accurately calculated from the approximate position of the target detected by radar and cameras. A joint classification network is used to extract micro-Doppler features from the time-frequency spectrum and texture features from images in the ROIs. A fusion dataset between radar and camera is established using a fusion data acquisition platform and includes intersections, highways, roads, and playgrounds in schools during the day and at night. The traditional radar signal algorithm, the Faster R-CNN model and our proposed fusion network model, called RCF-Faster R-CNN, are evaluated in this dataset. The experimental results indicate that the mAP(mean Average Precision) of our network is up to 89.42% more accurate than the traditional radar signal algorithm and up to 32.76% higher than Faster R-CNN, especially in the environment of low light and strong electromagnetic clutter.

show abstract

“…Another approach for addressing the issues of pedestrian recognition in blind areas is a remote sensing-based scheme using cameras and lidars [ 9 ]. Because these sensors can recognize different measurement areas, the sensor fusion approach to detect pedestrians in the blind area (occluded pedestrians) has been studied [ 10 ]. However, these techniques assumed that the pedestrians are partially visible and almost all studies do not assume the pedestrians that are completely invisible.…”

Section: Introductionmentioning

confidence: 99%

Pedestrian Detection in Blind Area and Motion Classification Based on Rush-Out Risk Using Micro-Doppler Radar

Hayashi

Saho

Daiki

et al. 2021

Sensors

View full text Add to dashboard Cite

Various remote sensing technologies have been applied in intelligent vehicles and robots for surrounding-environment recognition. However, these technologies experience difficulties in detecting pedestrians in blind areas and their motions, such as rush-out behaviors. To address this issue, we present a radar-based technique for the detection of pedestrians in blind areas and the classification of different risks of rush-out behaviors among detected pedestrians. We verify their ability to detect pedestrian motion in blind areas by conducting experiments in two environments with blind areas formed by outdoor cars and indoor walls. Then, the classification of motions with different risks of rush-out behaviors among pedestrians detected in the blind areas is demonstrated. We use the clustering method to accurately classify several types of behaviors with different rush-out risks in both environments.

show abstract

Occlusion aware sensor fusion for early crossing pedestrian detection

Cited by 24 publications

References 29 publications

CNN Based Road User Detection Using the 3D Radar Cube

CNN Based Road User Detection Using the 3D Radar Cube

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

Pedestrian Detection in Blind Area and Motion Classification Based on Rush-Out Risk Using Micro-Doppler Radar

Contact Info

Product

Resources

About