YOdar: Uncertainty-based Sensor Fusion for Vehicle Detection with Camera and Radar Sensors

Kowol, Kamil; Rottmann, Matthias; Bracke, Stefan; Gottschalk, Hanno

doi:10.5220/0010239301770186

Cited by 18 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This, along with the YOLOv3 network to characterize objects, was used for detecting vehicles at night. YOdar (YOLO radar) achieved a 39.4% mean average precision (mAP), compared to 31.36% mAP for YOLOv3 [12].…”

Section: Fig 1: Illustration Of Relationship Between Estimated True P...mentioning

confidence: 99%

“…𝑐 𝑡−1 + 𝑖 𝑡 . 𝑐̅ 𝑡 (11) 𝑜 𝑡 = 𝜎(𝑊 𝑜𝑥 𝑥 𝑡 + 𝑤 𝑜ℎ ℎ 𝑡−1 + 𝑏 𝑜 ) (12) ℎ 𝑡 = 𝑜 𝑡 . tanh (𝑐 𝑡 ) (13) Where 𝑡 is the recent time to predict, 𝑤 = [𝑊 𝑓𝑥 , 𝑊 𝑓ℎ , 𝑊 𝑐𝑥 , 𝑊 𝑜𝑥 , 𝑊 𝑜ℎ ] are the weight, and 𝑏 = [𝑏 𝑓 , 𝑏 𝑖 , 𝑏 𝑐 , 𝑏 𝑜 ] are the biases.…”

Section: Bayesian-lstmmentioning

confidence: 99%

See 1 more Smart Citation

Uncertainty-based Sensor Fusion Architecture using Bayesian-LSTM Neural Network

Geragersian

Petrunin

Guo

et al. 2023

AIAA SCITECH 2023 Forum

View full text Add to dashboard Cite

Uncertainty-based sensor management for positioning is an essential component in safe drone operations inside urban environments with large urban valleys. These canyons significantly restrict the Line-Of-Sight signal conditions required for accurate positioning using Global Navigation Satellite Systems (GNSS). Therefore, sensor fusion solutions need to be in place which can take advantage of alternative Positioning, Navigation and Timing (PNT) sensors such as accelerometers or gyroscopes to complement GNSS information. Recent stateof-art research has focused on Machine Learning (ML) techniques such as Support Vector Machines (SVM) that utilize statistical learning to provide an output for a given input. However, understanding the uncertainty of these predictions made by Deep Learning (DL) models can help improve integrity of fusion systems. Therefore, there is a need for a DL model that can also provide uncertainty-related information as the output. This paper proposes a Bayesian-LSTM Neural Network (BLSTMNN) that is used to fuse GNSS and Inertial Measurement Unit (IMU) data. Furthermore, Protection Level (PL) is estimated based on the uncertainty distribution given by the system. To test the algorithm, Hardware-In-the-Loop (HIL) simulation has been performed, utilizing Spirent's GSS7000 simulator and OKTAL-SE Sim3D to simulate GNSS propagation in urban canyons. SimSENSOR is used to simulate the accelerometer and gyroscope. Results show that Bayesian-LSTM provides the best fusion performance compared to GNSS alone, and GNSS/IMU fusion using EKF and SVM. Furthermore, regarding uncertainty estimates, the proposed algorithm can estimate the positioning boundaries correctly, with an error rate of 0.4% and with an accuracy of 99.6%.

show abstract

Section: Fig 1: Illustration Of Relationship Between Estimated True P...mentioning

confidence: 99%

Section: Bayesian-lstmmentioning

confidence: 99%

Uncertainty-based Sensor Fusion Architecture using Bayesian-LSTM Neural Network

Geragersian

Petrunin

Guo

et al. 2023

AIAA SCITECH 2023 Forum

View full text Add to dashboard Cite

show abstract

“…The results indicated that the detection effect of the fusion network was better than that of the image network only, and the authors concluded that further exploration of the optimized network structure was needed. In the 2020, YOdar proposed by Kowol et al [23], two networks are used to process image and radar data, and the results are used for joint prediction, which significantly improves the detection performance. In the process, the features of the data of the two sensors are not fused.…”

Section: B Object Detection Based On Radar and Image Fusionmentioning

confidence: 99%

MS-YOLO: Object Detection Based on YOLOv5 Optimized Fusion Millimeter-Wave Radar and Machine Vision

et al. 2022

View full text Add to dashboard Cite

Millimeter-wave radar and machine vision are both important means for intelligent vehicles to perceive the surrounding environment. Aiming at the problem of multisensor fusion, this paper proposes the object detection method of millimeter-wave radar and vision fusion. Radar and camera complement each other, and radar data fusion in machine vision network can effectively reduce the rate of missed detection under insufficient light conditions, and improve the accuracy of remote small object detection. The radar information is processed by mapping transformation neural network to obtain the mask map, so that radar information and visual information in the same scale. A multi-data source deep learning object detection network (MS-YOLO) based on millimeter-wave radar and vision fusion was proposed. Homemade datasets were used for training and testing. This maximized the use of sensor information and improved the detection accuracy under the premise of ensuring the detection speed. Compared with the original YOLOv5 (the fifth version of the You Only Look Once) network, the results show that the MS-YOLO network meets the accuracy requirements better. Among the models, the large model of MS-YOLO has the highest accuracy with an mAP reaching 0.888. The small model of MS-YOLO has good accuracy and speed, and the mAP reaches 0.841 while maintaining a high frame rate of 65 fps.

show abstract

“…LIDAR (Light Detection and Ranging) was used in the form of sensors attached to both vehicles and certain points of the road to detect oncoming vehicles [8]. Other non-intrusive methods like ASFF (Adaptive Spatial Feature Fusion) [9] and Radar Sensors [10] were also used. The interest in this field and its innovations date back to the 1970s [11].…”

Section: Literature Reviewmentioning

confidence: 99%

Performance Analysis of Deep Learning YOLO models for South Asian Regional Vehicle Recognition

Rafi¹,

Chakma²,

Mahmud³

et al. 2022

IJACSA

View full text Add to dashboard Cite

For years, humans have pondered the possibility of combining human and machine intelligence. The purpose of this research is to recognize vehicles from media and while there are multiple models associated with this, models that can detect vehicles commonly used in developing countries like Bangladesh, India, etc. are scarce. Our focus was to assimilate the largest dataset of vehicles exclusive to South Asia in addition to the more common universal vehicles and apply it to track and recognize these vehicles, even in motion. To develop this, we increased the class variations and quantity of the data and used multiple variations of the YOLOv5 model. We trained different versions of the model with our dataset to properly measure the degree of accuracy between the models in detecting the more unique vehicles. If vehicle detection and tracking are adopted and implemented in live traffic camera feeds, the information can be used to create smart traffic systems that can regulate congestion and routing by identifying and separating fast and slow-moving vehicles on the road. The comparison between the three different YOLOv5 models led to an analysis that indicates that the large variant of the YOLOv5 architecture outperforms the rest.

show abstract

YOdar: Uncertainty-based Sensor Fusion for Vehicle Detection with Camera and Radar Sensors

Cited by 18 publications

References 0 publications

Uncertainty-based Sensor Fusion Architecture using Bayesian-LSTM Neural Network

Uncertainty-based Sensor Fusion Architecture using Bayesian-LSTM Neural Network

MS-YOLO: Object Detection Based on YOLOv5 Optimized Fusion Millimeter-Wave Radar and Machine Vision

Performance Analysis of Deep Learning YOLO models for South Asian Regional Vehicle Recognition

Contact Info

Product

Resources

About