Reinforcement Learning for Ridesharing: An Extended Survey

Qin, Zhiwei; Zhu, Hongtu; Ye, Jieping

doi:10.48550/arxiv.2105.01099

Cited by 4 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the vehicle repositioning problems, we briefly summarize three key components for the framework design: (1) improvement on the policy-/value-based network (Jin et al 2019;Holler et al 2019;Wang et al 2020;Jiao et al 2021), (2) reward shaping (Tang et al 2019;Shou and Di 2020), and (3) state representation design (Al-Abbasi, Ghosh, and Aggarwal 2019;Tang et al 2019;Schmoll and Schubert 2020). Interested readers are referred to (Qin, Zhu, and Ye 2021) for a detailed review on RL in vehicle repositioning problem.…”

Section: Related Workmentioning

confidence: 99%

“…Sample complexity acts as one key challenge to enhance the process efficiency for a practical mechanism for vehicle repositioning (Qin, Zhu, and Ye 2021). To improve the sample complexity, one solution is to manually design the high-level policy using the domain knowledge of the ride-hailing system (e.g., pre-specifying the supply-demand distribution (Xu et al 2018)).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

Qian¹,

Guo²,

Aggarwal³

2021

Preprint

View full text Add to dashboard Cite

In a ride-hailing system, an optimal relocation of vacant vehicles can significantly reduce fleet idling time and balance the supply-demand distribution, enhancing system efficiency and promoting driver satisfaction and retention. Model-free deep reinforcement learning (DRL) has been shown to dynamically learn the relocating policy by actively interacting with the intrinsic dynamics in large-scale ride-hailing systems. However, the issues of sparse reward signals and unbalanced demand and supply distribution place critical barriers in developing effective DRL models. Conventional exploration strategy (e.g., the -greedy) may barely work under such an environment because of dithering in low-demand regions distant from high-revenue regions. This study proposes the deep relocating option policy (DROP) that supervises vehicle agents to escape from oversupply areas and effectively relocate to potentially underserved areas. We propose to learn the Laplacian embedding of a time-expanded relocation graph, as an approximation representation of the system relocation policy. The embedding generates task-agnostic signals, which in combination with task-dependent signals, constitute the pseudo-reward function for generating DROPs. We present a hierarchical learning framework that trains a highlevel relocation policy and a set of low-level DROPs. The effectiveness of our approach is demonstrated using a custombuilt high-fidelity simulator with real-world trip record data. We report that DROP significantly improves baseline models with 15.7% more hourly revenue and can effectively resolve the dithering issue in low-demand areas.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

Qian¹,

Guo²,

Aggarwal³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Note that in some literature the scenario of multiple passengers is also named as ridesharing. In this survey, we use ride-pooling specifically for disambiguation following [23].…”

Section: Ridesharingmentioning

confidence: 99%

“…or a multi-agent one. Previous literature investigating relative problems includes surveys by Haydari et al [22], Qin et al [23] and several reviews on VRP [24]. However, Haydari et al [22] focused on the general planning problems in Intelligent Transportation Systems from where Transportation Signal Control (TSC) and Autonomous Driving are emphasized.…”

Section: Introductionmentioning

confidence: 99%

“…However, Haydari et al [22] focused on the general planning problems in Intelligent Transportation Systems from where Transportation Signal Control (TSC) and Autonomous Driving are emphasized. Qin et al [23] only investigated the dispatching problems in ridesharing scenarios, and Mazyavkina et al [24] introduced DRL solutions on mathematical VRPs included in more general combinatorial optimizations. In contrast, we are the first to define DDS from a practical system level and classify specific research problems in several scenarios with DRL-based solutions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Zong¹,

Feng²,

Tong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent technology development brings the booming of numerous new Demand-Driven Services (DDS) into urban lives, including ridesharing, on-demand delivery, express systems and warehousing. In DDS, a service loop is an elemental structure, including its service worker, the service providers and corresponding service targets. The service workers should transport either humans or parcels from the providers to the target locations. Various planning tasks within DDS can thus be classified into two individual stages: 1) Dispatching, which is to form service loops from demand/supply distributions, and 2)Routing, which is to decide specific serving orders within the constructed loops. Generating high-quality strategies in both stages is important to develop DDS but faces several challenging. Meanwhile, deep reinforcement learning (DRL) has been developed rapidly in recent years. It is a powerful tool to solve these problems since DRL can learn a parametric model without relying on too many problem-based assumptions and optimize long-term effect by learning sequential decisions. In this survey, we first define DDS, then highlight common applications and important decision/control problems within. For each problem, we comprehensively introduce the existing DRL solutions, and further summarize them in https://github.com/tsinghua-fib-lab/DDS Survey. We also introduce open simulation environments for development and evaluation of DDS applications. Finally, we analyze remaining challenges and discuss further research opportunities in DRL solutions for DDS.

show abstract

Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand

Gammelli¹,

Yang²,

Harrison³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous Mobility-on-Demand (AMoD) systems represent an attractive alternative to existing transportation paradigms, currently challenged by urbanization and increasing travel needs. By centrally controlling a fleet of self-driving vehicles, these systems provide mobility service to customers and are currently starting to be deployed in a number of cities around the world. Current learning-based approaches for controlling AMoD systems are limited to the single-city scenario, whereby the service operator is allowed to take an unlimited amount of operational decisions within the same transportation system. However, real-world system operators can hardly afford to fully re-train AMoD controllers for every city they operate in, as this could result in a high number of poorquality decisions during training, making the single-city strategy a potentially impractical solution. To address these limitations, we propose to formalize the multi-city AMoD problem through the lens of meta-reinforcement learning (meta-RL) and devise an actorcritic algorithm based on recurrent graph neural networks. In our approach, AMoD controllers are explicitly trained such that a small amount of experience within a new city will produce good system performance. Empirically, we show how control policies learned through meta-RL are able to achieve near-optimal performance on unseen cities by learning rapidly adaptable policies, thus making them more robust not only to novel environments, but also to distribution shifts common in real-world operations, such as special events, unexpected congestion, and dynamic pricing schemes. CCS CONCEPTS• Computing methodologies → Reinforcement learning; Multitask learning; Neural networks.

show abstract

Reinforcement Learning for Ridesharing: An Extended Survey

Cited by 4 publications

References 34 publications

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand

Contact Info

Product

Resources

About