Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems

Gammelli, Daniele; Yang, Kun; Harrison, J.; Rodrigues, Filipe Almendagna; Pereira, Francisco Carlos; Pavone, Marco

doi:10.48550/arxiv.2104.11434

Cited by 10 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Autonomous vehicles are a type of more advanced robots with rich sensors, and have the capability to drive autonomously and safely on the road. Recent approaches applied GNN on connected autonomous vehicles (CAV) network, utilized spatiotemporal relations between CAVs [43,76]. In summary, GNNs can leverage the network structures in swarm and CAV networks communication and controls.…”

Section: Robotics and Autonomous Vehiclementioning

confidence: 99%

See 1 more Smart Citation

Graph Neural Networks in IoT: A Survey

Dong¹,

Tang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The Internet of Things (IoT) boom has revolutionized almost every corner of people's daily lives: healthcare, home, transportation, manufacturing, supply chain, and so on. With the recent development of sensor and communication technologies, IoT devices including smart wearables, cameras, smartwatches, and autonomous vehicles can accurately measure and perceive their surrounding environment. Continuous sensing generates massive amounts of data and presents challenges for machine learning. Deep learning models (e.g., convolution neural networks and recurrent neural networks) have been extensively employed in solving IoT tasks by learning patterns from multi-modal sensory data. Graph Neural Networks (GNNs), an emerging and fast-growing family of neural network models, can capture complex interactions within sensor topology and have been demonstrated to achieve state-of-the-art results in numerous IoT learning tasks. In this survey, we present a comprehensive review of recent advances in the application of GNNs to the IoT field, including a deep dive analysis of GNN design in various IoT sensing environments, an overarching list of public data and source code from the collected publications, and future research directions. To keep track of newly published works, we collect representative papers and their open-source implementations and create a Github repository at GNN4IoT. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; • Computing methodologies → Neural networks.

show abstract

Section: Robotics and Autonomous Vehiclementioning

confidence: 99%

“…Autonomous vehicle: as we mentioned in Section 4, interactions of intelligent entities are important in GNN sensing domain. Existing approaches model the interactions between agents/vehicles as graph nodes[59,76,115,138]. NGSIM US101[10], and I-80[5] are datasets that record vehicle trajectory on two different highways.…”

mentioning

confidence: 99%

Graph Neural Networks in IoT: A Survey

Dong¹,

Tang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Different works have approached the high-level problem of dispatching through cascaded learning (Fluri et al, 2019), both dispatching and relocation of drivers through centralised multi-agent management (Holler et al, 2019), or just the relocation (Lei et al, 2020). Finally, graph neural networks have most recently been used to address the fleet management problem (Gammelli et al, 2021) and the multiple traveling salesman problem (Kaempfer & Wolf, 2018;Hu et al, 2020).…”

Section: Resource Managementmentioning

confidence: 99%

Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environment

Carvalho¹,

Sengupta²

2022

Preprint

View full text Add to dashboard Cite

In a warehouse environment, tasks appear dynamically. Consequently, a task management system that matches them with the workforce too early (e.g., weeks in advance) is necessarily sub-optimal. Also, the rapidly increasing size of the action space of such a system consists of a significant problem for traditional schedulers. Reinforcement learning, however, is suited to deal with issues requiring making sequential decisions towards a long-term, often remote, goal. In this work, we set ourselves on a problem that presents itself with a hierarchical structure: the task-scheduling, by a centralised agent, in a dynamic warehouse multi-agent environment and the execution of one such schedule, by decentralised agents with only partial observability thereof. We propose to use deep reinforcement learning to solve both the high-level scheduling problem and the low-level multi-agent problem of schedule execution. Finally, we also conceive the case where centralisation is impossible at test time and workers must learn how to cooperate in executing the tasks in an environment with no schedule and only partial observability.

show abstract

“…RL frameworks have used Graph Neural Networks (GNNs) to learn policies for tasks including robot exploration [53], multi-agent coordination [54], and active learning [55]. Gammelli et al [56] use GNNs to learn vehicle routing policies over transportation networks. Wang et al [57] model multi-joint robots through graphical structures and use these graph representations to learn continuous control policies.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

Ravichandran¹,

Peng²,

Hughes³

et al. 2021

Preprint

View full text Add to dashboard Cite

Representations are crucial for a robot to learn effective navigation policies. Recent work has shown that mid-level perceptual abstractions, such as depth estimates or 2D semantic segmentation, lead to more effective policies when provided as observations in place of raw sensor data (e.g., RGB images). However, such policies must still learn latent three-dimensional scene properties from mid-level abstractions. In contrast, high-level, hierarchical representations such as 3D scene graphs explicitly provide a scene's geometry, topology, and semantics, making them compelling representations for navigation. In this work, we present a reinforcement learning framework that leverages high-level hierarchical representations to learn navigation policies. Towards this goal, we propose a graph neural network architecture and show how to embed a 3D scene graph into an agent-centric feature space, which enables the robot to learn policies for low-level action in an end-to-end manner. For each node in the scene graph, our method uses features that capture occupancy and semantic content, while explicitly retaining memory of the robot trajectory. We demonstrate the effectiveness of our method against commonly used visuomotor policies in a challenging object search task. These experiments and supporting ablation studies show that our method leads to more effective object search behaviors, exhibits improved long-term memory, and successfully leverages hierarchical information to guide its navigation objectives.

show abstract

Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems

Cited by 10 publications

References 17 publications

Graph Neural Networks in IoT: A Survey

Graph Neural Networks in IoT: A Survey

Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environment

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

Contact Info

Product

Resources

About