CoFF: Cooperative Spatial Feature Fusion for 3-D Object Detection on Autonomous Vehicles

Guo, Jingda; Carrillo, Dominic; Tang, Sihai; Chen, Qi; Yang, Qing; Fu, Song; Wang, Xi; Wang, Nannan; Palacharla, Paparao

doi:10.1109/jiot.2021.3053184

Cited by 28 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…a) Traditional Collaboration: At the early stage of the research on collaborative perception, people tend to use traditional strategies to fuse features. These intermediate collaboration applies the permutation invariant operations on deep features [42,55].…”

Section: A Improve Collaboration Efficiency and Performancementioning

confidence: 99%

“…Maxout strategy in F-Cooper [42] essentially keeps larger values of two feature maps and ignores smaller ones, which cannot capture the importance of weak features or enhance the weak feature. To address the above limitations, Guo et al [55] propose CoFF approach. In essence, CoFF weights the overlapped features by measuring their similarity and overlapping area.…”

Section: A Improve Collaboration Efficiency and Performancementioning

confidence: 99%

See 1 more Smart Citation

Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges

Han¹,

Zhang²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: A Improve Collaboration Efficiency and Performancementioning

confidence: 99%

Section: A Improve Collaboration Efficiency and Performancementioning

confidence: 99%

Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges

Han¹,

Zhang²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…On the other hand, Thandavarayan et al [1,3] has analyzed the performance of cooperative perception messages and offered generation rules to define which information should be included. In algorithm design, Chen et al [4,5] and Guo et al [35] have proposed a series of feature map fusion methods on CAVs by vehicle-to-vehicle (V2V) communication. Aoki et al [32] proposed the scheme with deep reinforcement learning (DRL) to select the data to transmit.…”

Section: Current Cooperative Perception Schemesmentioning

confidence: 99%

Pillar-Based Cooperative Perception from Point Clouds for 6G-Enabled Cooperative Autonomous Vehicles

Wang

Guo

Wang

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

3D object detection is a significant aspect of the perception module in autonomous driving; however, with current technology, data sharing between vehicles and cloud servers for cooperative 3D object detection under the strict latency requirement is limited by the communication bandwidth. The sixth-generation (6G) networks have accelerated the transmission rate of the sensor data significantly with extreme low-latency and high-speed data transmission. However, which sensor data format and when to transmit it are still challenging. To address these issues, this study proposes a cooperative perception framework combined with a pillar-based encoder and Octomap-based compression at edges for connected autonomous vehicles to reduce the amount of missing detection in blind spots and further distances. This approach satisfies the constraints on the accuracy of the task perception and provides drivers or autonomous vehicles with sufficient reaction time by applying fixed encoders to learn a representation of point clouds (LiDAR sensor data). Extensive experiment results show that the proposed approach outperforms the previous cooperative perception schemes running at 30 Hz, and the accuracy of the object bounding box results in further distances (greater than 12 m). Furthermore, this approach achieves a lower total delay for the procession of the fusion data and the transmission of the cooperative perception message. To the best of our knowledge, this study is the first to introduce a pillar-based encoder and Octomap-based compression framework for cooperative perception between vehicles and edges in connected autonomous driving.

show abstract

“…Hurl et al [22] proposed an end-to-end distributed perception model for cooperative perception from synthetic driving data. Guo et al [9] proposed a cooperative spatial feature fusion method for high-quality 3D object detection in autonomous vehicle systems. Arnold et al [8] introduced two cooperative 3D object detection schemes using single modality sensors.…”

Section: B Collaborative 3d Object Detectionmentioning

confidence: 99%

“…For exemple, in [19], a feature fusion based cooperative perception approach was proposed for 3D object detection on autonomous vehicles. In [9], a cooperative spatial feature fusion approach was proposed to effectively fuse feature maps for improving 3D object detection accuracy. However, transmitting feature maps across a fully-connected graph would bring high communication costs and delays, especially when the cross-agent bandwidth is limited.…”

Section: Introductionmentioning

confidence: 99%

Collaborative 3D Object Detection for Automatic Vehicle Systems via Learnable Communications

Wang¹,

Zhou²,

Gong³

2022

Preprint

View full text Add to dashboard Cite

Accurate detection of objects in 3D point clouds is a key problem in autonomous driving systems. Collaborative perception can incorporate information from spatially diverse sensors and provide significant benefits for improving the perception accuracy of autonomous driving systems. In this work, we consider that the autonomous vehicle uses local point cloud data and combines information from neighboring infrastructures through wireless links for cooperative 3D object detection. However, information sharing among vehicle and infrastructures in predefined communication schemes may result in communication congestion and/or bring limited performance improvement. To this end, we propose a novel collaborative 3D object detection framework that consists of three components: feature learning networks that map point clouds into feature maps; an efficient communication block that propagates compact and fine-grained query feature maps from vehicle to support infrastructures and optimizes attention weights between query and key to refine support feature maps; a region proposal network that fuses local feature maps and weighted support feature maps for 3D object detection. We evaluate the performance of the proposed framework using a synthetic cooperative dataset created in two complex driving scenarios: a roundabout and a T-junction. Experiment results and bandwidth usage analysis demonstrate that our approach can save communication and computation costs and significantly improve detection performance under different detection difficulties in all scenarios.

show abstract

CoFF: Cooperative Spatial Feature Fusion for 3-D Object Detection on Autonomous Vehicles

Cited by 28 publications

References 14 publications

Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges

Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges

Pillar-Based Cooperative Perception from Point Clouds for 6G-Enabled Cooperative Autonomous Vehicles

Collaborative 3D Object Detection for Automatic Vehicle Systems via Learnable Communications

Contact Info

Product

Resources

About