Large-Scale Order Dispatch in On-Demand Ride-Hailing Platforms

Xu, Zhe; Li, Zhixin; Guan, Qingwen; Zhang, Dingshui; Li, Qiang; Nan, Junxiao; Liu, Chunyang; Bian, Wei; Ye, Jieping

doi:10.1145/3219819.3219824

Cited by 306 publications

(219 citation statements)

References 23 publications

Supporting

Mentioning

216

Contrasting

Unclassified

Order By: Relevance

“…at is to say, the principle of matching orders based on distance is equivalent to random matching in our environment se ing. • MDP: Proposed by Xu et al [37], a planning and learning method based on decentralized multi-agent deep reinforcement learning and centralized combinatorial optimization.…”

Section: Methodsmentioning

confidence: 99%

“…But considering the current operational ride-sharing scenarios, it is hard to perform eet management for it is impossible to force drivers to designated regions. e mentioned MARL method [37] is an independent MARL method, which ignores the interactions between agents. However, it is a consensus to consider that the agent interactions have a positive impact on making optimal decisions.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Zhou

Jin

Zhang

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Improving the e ciency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicles and orders to be matched at every second which is of very high computational cost. In this paper, we propose a decentralized execution order-dispatching method based on multi-agent reinforcement learning to address the large-scale order-dispatching problem. Di erent from the previous cooperative multi-agent reinforcement learning algorithms, in our method, all agents work independently with the guidance from an evaluation of the joint policy since there is no need for communication or explicit cooperation between agents. Furthermore, we use KL-divergence optimization at each time step to speed up the learning process and to balance the vehicles (supply) and orders (demand). Experiments on both the explanatory environment and real-world simulator show that the proposed method outperforms the baselines in terms of accumulated driver income (ADI) and Order Response Rate (ORR) in various tra c environments. Besides, with the support of the online platform of Didi Chuxing, we designed a hybrid system to deploy our model. CCS CONCEPTS•Computing methodologies → Multi-agent reinforcement learning; • eory of computation → Multi-agent reinforcement learning; •Applied computing → Transportation;

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Zhou

Jin

Zhang

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

show abstract

“…One example of a heuristic solution to the order dispatching problem is the myopic pickup distance minimization (MPDM), which ignores temporal dynamics and always assigns the closest available driver to a requesting order [12]. In Local Policy Improvement [13], handcrafted heuristics are combined with a machine learning method by summarizing supply and demand patterns into a table and then using the learned patterns to account for future gains in the real-time planner of the dispatching solution. A fully machine learning based approach to the dispatching problem is presented in [14], where deep Q-learning is used for learning dispatching strategies from the perspective of a single driver.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Multi-driver Vehicle Dispatching and Repositioning Problem

Holler

Vuorio

Qin³

et al. 2019

2019 IEEE International Conference on Data Mining (ICDM)

Self Cite

View full text Add to dashboard Cite

Order dispatching and driver repositioning (also known as fleet management) in the face of spatially and temporally varying supply and demand are central to a ride-sharing platform marketplace. Hand-crafting heuristic solutions that account for the dynamics in these resource allocation problems is difficult, and may be better handled by an end-to-end machine learning method. Previous works have explored machine learning methods to the problem from a high-level perspective, where the learning method is responsible for either repositioning the drivers or dispatching orders, and as a further simplification, the drivers are considered independent agents maximizing their own reward functions. In this paper we present a deep reinforcement learning approach for tackling the full fleet management and dispatching problems. In addition to treating the drivers as individual agents, we consider the problem from a system-centric perspective, where a central fleet management agent is responsible for decisionmaking for all drivers.

show abstract

“…One alternative is to model each available vehicle as an agent [21, Forthcoming Order Figure 1: Ride-hailing task in thermodynamics view. 37,39]. However, such setting needs to maintain thousands of agents interacting with the environment, which brings a huge computational cost.…”

Section: Introductionmentioning

confidence: 99%

“…Greedily matching vehicles with long-distance orders can receive high immediate gain at a single order dispatching stage, but it would harm order response rate (ORR) and future revenue especially during rush hour because of its long drive time and unpopular destination. Recent attempts [21,37,39] deployed RL to combine instant order reward from online planning with future state-value as the final matching value. However, the coordination between different regions is still far from optimal.…”

Section: Introductionmentioning

confidence: 99%

CoRide

Jin

Zhou

Zhang

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

How to optimally dispatch orders to vehicles and how to trade off between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decisionmaking task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multihead attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of citywide hybrid order dispatching and fleet management over strong baselines.

show abstract

Large-Scale Order Dispatch in On-Demand Ride-Hailing Platforms

Cited by 306 publications

References 23 publications

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Deep Reinforcement Learning for Multi-driver Vehicle Dispatching and Repositioning Problem

CoRide

Contact Info

Product

Resources

About