Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

Abstract

The development of autonomous vehicles and unmanned aerial vehicles has led to a current research focus on improving the environmental perception of automation equipment. The unmanned platform detects its surroundings and then makes a decision based on environmental information. The major challenge of environmental perception is to detect and classify objects precisely; thus, it is necessary to perform fusion of different heterogeneous data to achieve complementary advantages. In this paper, a robust object detection and classification algorithm based on millimeter-wave (MMW) radar and camera fusion is proposed. The corresponding regions of interest (ROIs) are accurately calculated from the approximate position of the target detected by radar and cameras. A joint classification network is used to extract micro-Doppler features from the time-frequency spectrum and texture features from images in the ROIs. A fusion dataset between radar and camera is established using a fusion data acquisition platform and includes intersections, highways, roads, and playgrounds in schools during the day and at night. The traditional radar signal algorithm, the Faster R-CNN model and our proposed fusion network model, called RCF-Faster R-CNN, are evaluated in this dataset. The experimental results indicate that the mAP(mean Average Precision) of our network is up to 89.42% more accurate than the traditional radar signal algorithm and up to 32.76% higher than Faster R-CNN, especially in the environment of low light and strong electromagnetic clutter.

Improved single shot multibox detector target detection method based on deep feature fusion

“…The detection target scale range has certain limitations, so it is prone to inaccurate positioning and high classification error rate. 58,59 Before fusing the features of different scales in each layer, it is necessary to ensure that the resolution of the feature map is consistent. Taking the feature map conv4_3 as an example, the processing methods of the adjacent features conv3_3 and conv5_3 of conv4_3 are further explained.…”

Space Target Classification Improvement by Generating Micro-Doppler Signatures Considering Incident Angle

“…However, there is also a disadvantage in that the explainability of the classification result is low. Classification using micro-Doppler signatures has been currently being studied more actively for application to the motion of drones and humans than to application to space targets [19][20][21][22][23][24]. This is because drones and radar technologies have been expanded to the private sector and are attracting attention.…”

3DRIED: A High-Resolution 3-D Millimeter-Wave Radar Dataset Dedicated to Imaging and Evaluation

“…In the near-field MMW imaging field, Wang et al presented a novel 3-D microwave sparse reconstruction method based on a complex-valued sparse reconstruction network (CSR-Net) [30], a novel range migration kernel-based iterative-shrinkage thresholding network (RMIST-Net) [31], and a lightweight FISTA-Inspired Sparse Reconstruction Network for MMW 3-D Holography [32]. In the aspect of application, a detection and classification algorithm based on the MMW radar and camera fusion is proposed in [33]; Cui et al [34] presented a K-means-based machine learning algorithm for user clustering with MMW system; Ref. [35] demonstrates that MMW can be used for robust gesture recognition and can track gestures; A unified framework of multiple target detection, recognition, and fusion is proposed in [36].…”