Latency-Aware Collaborative Perception

Lei, Zixing; Ren, Shunli; Hu, Yue; Zhang, Wenjun; Chen, Siheng

doi:10.1007/978-3-031-19824-3_19

Cited by 47 publications

(11 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our approach overcomes the challenge of temporal asynchrony and reduces transmission costs. We propose a feature flow prediction method that is different from (Lei et al, 2022), which focuses on integrating per-frame features from other vehicles. Our approach addresses the challenge of temporal asynchrony by predicting future feature and compensating for the latency, resulting in improved detection performance.…”

Section: Related Workmentioning

confidence: 99%

“…Another possible solution to solve the temporal asynchrony problem is proposed in (Lei et al, 2022), which generates future features with received historical features on vehicle devices. However, this solution requires significant computing resources to process the historical frames and extract temporal correlations to predict future features.…”

Section: E Relationship To Other Existing Possible Solutionsmentioning

confidence: 99%

See 1 more Smart Citation

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

Yu¹,

Tang²,

Xie³

et al. 2023

Preprint

View full text Add to dashboard Cite

Cooperatively utilizing both ego-vehicle and infrastructure sensor data can significantly enhance autonomous driving perception abilities. However, temporal asynchrony and limited wireless communication in traffic environments can lead to fusion misalignment and impact detection performance. This paper proposes Feature Flow Net (FFNet), a novel cooperative detection framework that uses a feature flow prediction module to address these issues in vehicle-infrastructure cooperative 3D object detection. Rather than transmitting feature maps extracted from still-images, FFNet transmits feature flow, which leverages the temporal coherence of sequential infrastructure frames to predict future features and compensate for asynchrony. Additionally, we introduce a self-supervised approach to enable FFNet to generate feature flow with feature prediction ability. Experimental results demonstrate that our proposed method outperforms existing cooperative detection methods while requiring no more than 1/10 transmission cost of raw data on the DAIR-V2X dataset when temporal asynchrony exceeds 200ms. The code is available at https://github.com/haibao-yu/FFNet-VIC3D.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: E Relationship To Other Existing Possible Solutionsmentioning

confidence: 99%

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

Yu¹,

Tang²,

Xie³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Prior works have studied cooperative vehicular perception using lidar sensors [13]- [17] where the association problem is significantly easier. Furthermore, other works present methodology which does not require machine learning and relies only on the position estimates of surrounding objects to create associations.…”

Section: Related Workmentioning

confidence: 99%

Multi-Source Feature Fusion for Object Detection Association in Connected Vehicle Environments

2022

View full text Add to dashboard Cite

Improvements in vehicular perception systems over the last decade have enabled new levels of safety and awareness in modern production vehicles. However, achievable performance of these perception systems is bounded by sensor limitations, such as range, and environmental factors, such as occlusion. Collaborative perception circumvents these limitations by incorporating sensor data from multiple sources to fill in perception gaps experienced by an individual sources' sensors. This paper explores one important aspect of collaborative perception: simultaneously associating objects detected by multiple individual vehicles with each other. This task is crucial as the inability to perform such object association accurately can lead to duplicate or missed detections, which can lead to unsafe driving behavior. This work proposes a graph neural network model for this task that achieves an average precision (AP) of 0.882 in a challenging virtual environment consisting of 25 unique, simultaneous, and mobile viewpoints. A simpler real-world scenario with two static viewpoints is also evaluated where the model achieves an AP of 0.998, showing that this model can readily transfer to real-world scenarios as well.

show abstract

“…Cooperative Perception enables multiple agents to perceive surrounding environment collaboratively by sharing their observations and knowledge, which offers a great potential for improving individual's safety, resilience and adaptability [13,26,8,5,15]. Recent years have witnessed many related systems developed to support a broad range of real-world applications, e.g., vehicle-to-vehicle (V2V)communication-aided autonomous driving [37,38,33,2,3,5], multirobot warehouse automation system [17,41] and multiple unmanned aerial vehicles (UAVs) for search and rescue [29].…”

Section: Related Workmentioning

confidence: 99%

“…Apart from performance, the robustness of system is also critical. Some methods [13,31,26] address problems arising from the communication process, for example, latency [13,37], localization error [31,40], and communication interruption [26]. Some other approaches [16,4] investigates selfsupervised learning mechanisms to improve the generalization ability of collaboration models.…”

Section: Related Workmentioning

confidence: 99%

Cooperative control method of multi-missile formation under uncontrollable speed

Zhang

Lyu

et al. 2021

西北工业大学学报

View full text Add to dashboard Cite

Under the missile speed is uncontrollable, a design method of multi-missile formation flight controller based on the sliding mode variable structure control theory and adaptive dynamic surface control theory is proposed. Firstly, according to the relative position of the leader and the follower in the inertial frame, the tracking error model for the relative position and the expected relative position between the leader and the follower is obtained, and the multi-missile formation control system in the inertial coordinate system is obtained. Secondly, in order to obtain the expression of the formation control system in the ballistic coordinate system, the acceleration of the missile in the ballistic coordinate system is converted to the inertial coordinate system. Combining with the tracking of the relative position and the desired relative position of the leader and the followers, we can obtain the simplified error model for the formation control system. Then the sliding mode variable structure control theory and the adaptive dynamic surface control theory are used to design the formation controllers for the leader and follower missiles respectively, and the stability of the present controller is analysed via the Lyapunov stability theory. Finally, the designed formation controllers are used for the leader and follower missiles to simulate the parameters. The results verify the feasibility and effectiveness of the present method.

show abstract

Latency-Aware Collaborative Perception

Cited by 47 publications

References 28 publications

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

Multi-Source Feature Fusion for Object Detection Association in Connected Vehicle Environments

Cooperative control method of multi-missile formation under uncontrollable speed

Contact Info

Product

Resources

About