Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplace

Eshkevari, Soheil Sadeghi; Tang, Xiaocheng; Qin, Zhiwei; Mei, Jinhan; Zhang, Cheng; Meng, Qianying; Xu, Jing

doi:10.1145/3534678.3539095

Cited by 7 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the efficiency of dispatching and rebalancing depends on future ride requests, model-predictive control (Alonso-Mora et al 2017) and stochastic model-predictive control algorithms (Tsao et al 2018) have been developed to incorporate information about future ride requests. A complementary stream of research studied similar problems from a deep reinforcement learning (DRL) perspective, explicitly focusing on rebalancing (Jiao et al 2021, Gammelli et al 2021, Skordilis et al 2022, Liang et al 2022 or non-myopic (anticipative) dispatching (Xu et al 2018, Li et al 2019, Tang et al 2019, Zhou et al 2019, Sadeghi Eshkevari et al 2022, Enders et al 2022.…”

Section: Related Workmentioning

confidence: 99%

Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control

Jungel¹,

Parmentier²,

Schiffer³

et al. 2023

Preprint

View full text Add to dashboard Cite

Autonomous mobility-on-demand systems are a viable alternative to mitigate many transportation-related externalities in cities, such as rising vehicle volumes in urban areas and transportation-related pollution. However, the success of these systems heavily depends on efficient and effective fleet control strategies. In this context, we study online control algorithms for autonomous mobility-on-demand systems and develop a novel hybrid combinatorial optimization enriched machine learning pipeline which learns online dispatching and rebalancing policies from optimal full-information solutions. We test our hybrid pipeline on large-scale realworld scenarios with different vehicle fleet sizes and various request densities. We show that our approach outperforms state-of-the-art greedy, and model-predictive control approaches with respect to various KPIs, e.g., by up to 17.1% and on average by 6.3% in terms of realized profit.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control

Jungel¹,

Parmentier²,

Schiffer³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to the significance of order dispatching, it has attracted lots of interest from both industry and academia, and a plethora of works (Zhang et al 2017;Xu et al 2018;Tang et al 2021;Sadeghi Eshkevari et al 2022;Sun et al 2022;Jiang et al 2023;Qin, Zhu, and Ye 2021;Zhou et al 2019;Xi et al 2022;Si et al 2023;Wang et al 2023a,b)have been conducted to address this problem. Especially, with the recent advance in deep learning and computing power, deep reinforcement learning (DRL) has shown great potential for sequential decision-making problems such as ridehailing order dispatching.…”

Section: Introductionmentioning

confidence: 99%

“…Especially, with the recent advance in deep learning and computing power, deep reinforcement learning (DRL) has shown great potential for sequential decision-making problems such as ridehailing order dispatching. For example, (Sadeghi Eshkevari et al 2022) proposed a standalone DRL-based dispatching strategy that is equipped with multiple novel mechanisms to ensure robust and efficient on-policy learning and inference while being adaptable for full-scale deployment. (Tang et al 2021) designed a value function and hierarchical DRL to unify order dispatching and vehicle relocation problems.…”

Section: Introductionmentioning

confidence: 99%

NondBREM: Nondeterministic Offline Reinforcement Learning for Large-Scale Order Dispatching

Zhang,

Wang,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

One of the most important tasks in ride-hailing is order dispatching, i.e., assigning unserved orders to available drivers. Recent order dispatching has achieved a significant improvement due to the advance of reinforcement learning, which has been approved to be able to effectively address sequential decision-making problems like order dispatching. However, most existing reinforcement learning methods require agents to learn the optimal policy by interacting with environments online, which is challenging or impractical for real-world deployment due to high costs or safety concerns. For example, due to the spatiotemporally unbalanced supply and demand, online reinforcement learning-based order dispatching may significantly impact the revenue of the ride-hailing platform and passenger experience during the policy learning period. Hence, in this work, we develop an offline deep reinforcement learning framework called NondBREM for large-scale order dispatching, which learns policy from only the accumulated logged data to avoid costly and unsafe interactions with the environment. In NondBREM, a Nondeterministic Batch-Constrained Q-learning (NondBCQ) module is developed to reduce the algorithm extrapolation error and a Random Ensemble Mixture (REM) module that integrates multiple value networks with multi-head networks is utilized to improve the model generalization and robustness. Extensive experiments on large-scale real-world ride-hailing datasets show the superiority of our design.

show abstract

“…There are extensive works in the ridesharing literature on marketplace optimization to improve a rideshare platform's performance, typically in market equilibrium and efficiency. Some recent works cover algorithms for dispatch [9,12,24,25,29,34], repositioning [5,7,15,18,20], pricing [2,8,14,19,30], and incentives [23,28]. There have also been works focusing on investigations and analyses of the driving factors for rideshare market efficiency [1,11,13,17].…”

Section: Introductionmentioning

confidence: 99%

A Unified Representation Framework for Rideshare Marketplace Equilibrium and Efficiency

Chin¹,

Qin²

2023

Preprint

View full text Add to dashboard Cite

Ridesharing platforms are a type of two-sided marketplace where "supply-demand balance" is critical for market efficiency and yet is complex to define and analyze. We present a unified analytical framework based on the graph-based equilibrium metric (GEM) for quantifying the supply-demand spatiotemporal state and efficiency of a ridesharing marketplace. GEM was developed as a generalized Wasserstein distance between the supply and demand distributions in a ridesharing market and has been used as an evaluation metric for algorithms expected to improve supply-demand alignment. Building upon GEM, we develop SD-GEM, a dual-perspective (supplyand demand-side) representation of rideshare market equilibrium. We show that there are often disparities between the two views and examine how this dual-view leads to the notion of market efficiency, in which we propose novel statistical tests for capturing improvement and explaining the underlying driving factors.

show abstract

Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplace

Cited by 7 publications

References 10 publications

Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control

Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control

NondBREM: Nondeterministic Offline Reinforcement Learning for Large-Scale Order Dispatching

A Unified Representation Framework for Rideshare Marketplace Equilibrium and Efficiency

Contact Info

Product

Resources

About