Traffic vehicle cognition in severe weather based on radar and infrared thermal camera fusion

Wang, Zhangu; Zhan, Jianming; Chuanguang, Duan; Guan, Xin; Yang, Kai

doi:10.1088/1361-6501/abfe34

Cited by 13 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following the approach of, we select the center of gravity of the 3D reference points as the first control point see equation (4). and obtain a set of vectors A.…”

Section: Extrinsic Parameter Regressionmentioning

confidence: 99%

See 1 more Smart Citation

A Target-based co-calibration framework for 3DRadar-camera using a modified corner reflector

Chen,

Shao,

Zhang

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

Most intelligent transportation and autonomous driving systems use the combination of millimeter-wave (MMW) radar and camera to achieve strong perception, and correct extrinsic calibration is a prerequisite for sensor fusion. Most prior calibration methods rely on specific calibration environment, resulting in low calibration efficiency, and are unsuitable for practical scenarios. This paper proposes a progressive and efficient extrinsic calibration method for 3D MMW radar and camera, which only requires testers to walk around the testing range with the calibration target, and the progressive process of extrinsic parameters regression is visualized. The proposed method has been tested on the View-of-Delft dataset and in the real world, and the results show that the method proposed in this paper can converge the calibrated extrinsic parameters quickly and has strong robustness to the noise present during the testing process. Compared to EPNP, LM, P3P (based on RANSAC), and LM (based on RANAC), our proposed calibration method demonstrates a smaller re-projection error and higher accuracy in terms of extrinsic parameters. All results indicate that our calibration method has good accuracy and efficiency for practical calibration scenarios.

show abstract

“…Following the approach of, we select the center of gravity of the 3D reference points as the first control point see equation (4). and obtain a set of vectors A.…”

Section: Extrinsic Parameter Regressionmentioning

confidence: 99%

“…dusty, foggy, rainy, and snowy) [3]. MMW radar is more robust to weather conditions than LIDAR [4,5]. Especially, 3D MMW radar can provide high angular resolution, and recently, low-cost 3D radar sensors have become available.…”

Section: Introductionmentioning

confidence: 99%

A Target-based co-calibration framework for 3DRadar-camera using a modified corner reflector

Chen,

Shao,

Zhang

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…The echo characteristics are apparent because the RCS on the front of the human body is more prominent than on other sides. Therefore, it is necessary to calculate the proportion of the return energy of the extremities and calculate the energy ratio according to equation (1). To prevent the influence of the trunk on the calculation of the energy ratio of the extremities, the coefficient ε is set to 500.…”

Section: Experimental Analysismentioning

confidence: 99%

“…Millimeter-wave radar has the advantages of small size, high bandwidth, and suitability for all-day and all-weather use. It is widely employed in unmanned vehicle detection [1][2][3], medical life signs detection [4], human behavior detection [5], and human-computer interaction technology's micro-action detection [6]. Compared with non-contact sensors (infrared [7], optical camera [8], etc), which detect human behavior, radio frequency signals emitted and received by radar can * Author to whom any correspondence should be addressed.…”

Section: Introductionmentioning

confidence: 99%

Application of multi-angle millimeter-wave radar detection in human motion behavior and micro-action recognition

Zhang¹,

Meng²,

Song³

et al. 2022

Meas. Sci. Technol.

View full text Add to dashboard Cite

Millimeter-wave radar is widely used in family safety, rehabilitation, and assisted living due to its ability to work all-weather and all day. Aiming at the problem that radar detection angle significantly impacts human behavior recognition, a recognition method based on multi-angle radar observation is adopted. We proposed a novel radar selection method called the energy domain ratio method (EDRM) to choose a radar with more sensitive features. Then, a Local tangent space alignment (LTSA) and an adaptive extreme learning machine (AELM) are presented to enhance the recognition rate of the model in a high noise environment. A multi-angle entropy (ME) feature and an improved extreme learning machine (IELM) are developed to identify human micro-motion in a low noise indoor environment. The effect of observation distance on the recognition effect was also explored. Experimental results show that the proposed model has a more than 86 percent recognition rate for human behavior in outdoor scenes and a recognition accuracy of more than 98 percent for indoor micro-action.

show abstract

“…Xue et al 25 designed a fast saliency map of vehicle targets in infrared images at night to achieve rapid acquisition of regions of interest (ROIs) for vehicles. Zhangu et al 29 fused radar and infrared cameras to greatly enrich the completeness of vehicle cognitive information and then proposed an attention mechanism based on radar guidance information to extract the vehicle ROI. Zhao et al 30 introduced a residual structure to fully retain the shallow detail information of small infrared target vehicles.…”

Section: Introductionmentioning

confidence: 99%

ComS-YOLO: a combinational and sparse network for detecting vehicles in aerial thermal infrared images

Zhang,

2024

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

Vehicle detection using aerial thermal infrared images has received significant attention because of its strong capability for day and night observations to supply information for vehicle tracking, traffic monitoring, and road network planning. Compared with aerial visible images, aerial thermal infrared images are not sensitive to lighting conditions. However, they have low contrast and blurred edges. Therefore, a combinational and sparse you-only-look-once (ComS-YOLO) neural network is put forward to accurately and quickly detect vehicles in aerial thermal infrared images. Therein, we adjust the structure of the deep neural network to balance the detection accuracy and running time. In addition, we propose an objective function that utilizes the diagonal distance of the corresponding minimum external rectangle, which prevents non-convergence when there is an inclusion relationship between the prediction and true boxes or in the case of width and height alignment. Furthermore, to avoid over-fitting in the training stage, we eliminate some redundant parameters via constraints and on-line pruning. Finally, experimental results on the NWPU VHR-10 and DARPA VIVID datasets show that the proposed ComS-YOLO network effectively and efficiently identifies the vehicles with a low missed rate and false detection rate. Compared with the Faster R-CNN and a series of YOLO neural networks, the proposed neural network presents satisfactory and competitive results in terms of the detection accuracy and running time. Furthermore, vehicle detection experiments under different environments are also carried out, which shows that our method can achieve an excellent and desired performance on detection accuracy and robustness of vehicle detection.

show abstract

Traffic vehicle cognition in severe weather based on radar and infrared thermal camera fusion

Cited by 13 publications

References 30 publications

A Target-based co-calibration framework for 3DRadar-camera using a modified corner reflector

A Target-based co-calibration framework for 3DRadar-camera using a modified corner reflector

Application of multi-angle millimeter-wave radar detection in human motion behavior and micro-action recognition

ComS-YOLO: a combinational and sparse network for detecting vehicles in aerial thermal infrared images

Contact Info

Product

Resources

About