Learning to Operate a Fleet of Cars

Abstract

In a mobility-on-demand system, travel requests are handled by a fleet of shared vehicles in an on-demand fashion. An important factor that determines the operational efficiency and service level of such a mobility-on-demand system is its operational policy that assigns available vehicles to open passenger requests and relocates idle vehicles. Previously described operational policies are based on control theoretical approaches, most notably on receding horizon control. In this work, we employ reinforcement learning techniques to design an operational policy for a mobility-on-demand system. In particular, we propose a cascaded learning framework to reduce the number of state-action pairs which allows for more efficient learning. We train our model using the AMoDeus simulation environment and real taxi trip travel data from the city of San Francisco. Finally, we demonstrate that our reinforcement learning based operational policy for mobility-on-demand systems outperforms state-of the art fleet operational policies that are based on conventional control theoretical approaches.

Simulation-Based Assessment of Parking Constraints for Automated Mobility on Demand: A Case Study of Zurich

“…These different changes in waiting times with different parking strategies and distributions can also be explained as follows. While typically in fleet control vehicles are rebalanced to locations of future anticipated demand [33,34,52,53] no rebalancing logic is used on top of the basic global bipartite matching in the present case. Idle and staying vehicles are merely sent to available parking spaces by the parking operating policies.…”

Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems

“…2) Cascaded Q-Learning (CQL) [10]: rebalancing policies are learned through tabular Q learning where the current distribution of idle vehicles is chosen as the sole representation of the state space. To achieve a more scalable state-action size, this approach introduces a hierarchical representation of the transportation network, thus learning behavior policies from coarse to finer levels.…”

“…As illustrated in Figure 2, we adopt a three-step decisionmaking framework similar to [10] to control an AMoD fleet:…”

“…The third and most relevant category for this work employs learning-based approaches to devise efficient algorithms without significantly compromising optimality [7]- [10]. Gueriau et al [8] developed RL-based decentralized approaches where the action of each vehicle is determined independently through a Q-learning policy.…”

“…Nevertheless, the action of only one vehicle is determined at each time step due to computation tractability, which might lead to myopic solutions. Fluri et al [10] developed a cascaded Q-learning approach to operate AMoD systems in a centralized manner. The cascaded structure significantly reduces the number of state-action pairs, allowing more efficient learning.…”