DeepFreight: A Model-free Deep-reinforcement-learning-based Algorithm for Multi-transfer Freight Delivery

Abstract

With the freight delivery demands and shipping costs increasing rapidly, intelligent control of fleets to enable efficient and cost-conscious solutions becomes an important problem. In this paper, we propose DeepFreight, a model-free deep-reinforcement-learning-based algorithm for multi-transfer freight delivery, which includes two closely-collaborative components: truck-dispatch and package-matching. Specifically, a deep multi-agent reinforcement learning framework called QMIX is leveraged to learn a dispatch policy, with which we can obtain the multi-step joint vehicle dispatch decisions for the fleet with respect to the delivery requests. Then an efficient multi-transfer matching algorithm is executed to assign the delivery requests to the trucks. Also, DeepFreight is integrated with a Mixed-Integer Linear Programming optimizer for further optimization. The evaluation results shows that the proposed system is highly scalable and ensures a 100% delivery success while maintaining low delivery-time and fuel consumption.

Learning to Send Reinforcements: Coordinating Multi-Agent Dynamic Police Patrol Dispatching and Rescheduling via Reinforcement Learning

“…To solve the problem, the authors decompose the problem into two stages: truck-dispatch and request-matching and leverage on QMIX to learn the dispatch policy while implement a separate matching algorithm for the second stage. Similarly, Chen et al [2022] solve a same-day delivery problem with vehicles and drones by decomposing the problem into two stages: learning assignment policy via DQN and rerouting via heuristic. Meanwhile, decomposes a dynamic courier dispatch problem into two stages namely dispatch and routing stage.…”

“…Communication. To coordinate amongst agents, various works focus on the aspect of learning to communicate in cooperative setting (see [Jiang and Lu, 2018;Das et al, 2019;). However, communication alone does not guarantee coordination especially when agents act simultaneously [Ruan et al, 2022].…”

“…Patrol Presence. A study by Doyle et al [2016] has shown that patrol presence invokes feelings of safety in people. We define patrol presence as a function of the number of effective time periods each patrol area is being patrolled in a shift.…”

“…• MADQN-BR-H. This is the two-stage approach similar to Chen et al [2021]. In order to evaluate solely on the joint vs. two-stage learning mechanisms, we retain the same components, BR and H. Phase 2.…”

“…However, current Multi-Agent RL (MARL) approaches deal with problems with simple actions (either discrete or continuous). Thus, to solve problems with complex action similar to MADPRP, current MARL approaches either decompose the problem into two stages (see [Chen et al, 2021]) and/or discretize the actions (see ).…”