Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshaking

Abstract

In this paper, we address bandwidth-limited and obstruction-prone collaborative perception, specifically in the context of multi-agent semantic segmentation. This setting presents several key challenges, including processing and exchanging unregistered robotic swarm imagery. To be successful, solutions must effectively leverage multiple non-static and intermittently-overlapping RGB perspectives, while heeding bandwidth constraints and overcoming unwanted foreground obstructions. As such, we propose an end-to-end learn-able Multi-Agent Spatial Handshaking network (MASH) to process, compress, and propagate visual information across a robotic swarm. Our distributed communication module operates directly (and exclusively) on raw image data, without additional input requirements such as pose, depth, or warping data. We demonstrate superior performance of our model compared against several baselines in a photo-realistic multi-robot AirSim environment, especially in the presence of image occlusions. Our method achieves an absolute 11% IoU improvement over strong baselines.

Perception Methods for Adverse Weather Based on Vehicle Infrastructure Cooperation System: A Review

“…Arnold et al [ 92 ] adaptably chose to share high-level information (perception results) or low-level information (raw perception data) according to the density and visibility of object information obtained from sensors. Glaser et al [ 124 ] utilized neural networks to learn the corresponding relationship among perception information from multiple ICVs and discarded pose information from other intelligent agents. Cui et al [ 125 ] constructed an end-to-end feature transformation learning model to achieve cooperative perception in V2V.…”

Collaborative Perception for Autonomous Driving: Current Status and Future Trend

“…They proposed end-to-end learnable neural reasoning layers that learn to estimate pose errors to make vehicles reach a consensus about those errors. [16] leveraged neural layers to learn the data correspondence without the requirements of pose information of other agents.…”

“…of 3D scenes from several agents. [23,24,16] focus on collaborative semantic segmentation with limited communication bandwidth. Figure .4(b) illustrates the effectiveness of collaborative semantic segmentation leveraging multi views in the presence of an unexpected obstruction.…”