Learning to Solve 3-D Bin Packing Problem via Deep Reinforcement Learning and Constraint Programming

Abstract

The Bin Packing Problem (BPP) has attracted enthusiastic research interest recently, owing to widespread applications in logistics and warehousing environments. It is truly essential to optimize the bin packing to enable more objects to be packed into boxes. Object packing order and placement strategy are the two crucial optimization objectives of the BPP. However, existing optimization methods for BPP, such as the genetic algorithm (GA), emerge as the main issues in highly computational cost and relatively low accuracy, making it difficult to implement in realistic scenarios. To well relieve the research gaps, we present a novel optimization methodology of two-dimensional (2D)-BPP and three-dimensional (3D)-BPP for objects with regular shapes via deep reinforcement learning (DRL), maximizing the space utilization and minimizing the usage number of boxes. First, an end-to-end DRL neural network constructed by a modified Pointer Network consisting of an encoder, a decoder and an attention module is proposed to achieve the optimal object packing order. Second, conforming to the top-down operation mode, the placement strategy based on a height map is used to arrange the ordered objects in the boxes, preventing the objects from colliding with boxes and other objects in boxes. Third, the reward and loss functions are defined as the indicators of the compactness, pyramid, and usage number of boxes to conduct the training of the DRL neural network based on an on-policy actor-critic framework.Finally, a series of experiments are implemented to compare our method with conventional packing methods, from which we conclude that our method outperforms these packing methods in both packing accuracy and efficiency.

A review on learning to solve combinatorial optimisation problems in manufacturing

“…Please note that the above problem description and MDP is for a basic offline 3D bin packing. The instantiations of the MDP have been successfully applied for specific packing problems in some works [20,66]. Other variants of packing problems and their corresponding MDPs (e.g.…”

“…Similarly, Jiang et al. design a sequence‐to‐sequence neural network with three decoders to be trained as three policies, and in the encoder a CNN is used for embedding the status of bins along with an attention mechanism for embedding the items [20]. In addition, Yang et al.…”

“…Large (continuous) action space: Compared to scheduling and routing problems, the action space in BPP is the combinations of items, positions and orientations. However, very a few current works learn to decide on all three aspects [20,66], since it will cause the issue of the large action space. This could be more intractable when continuous positions are considered [18].…”

An empirically efficient two-phase heuristic algorithm for two-dimensional bin packing problem

“…A lifelong learning hyper-he uristic method for bin packing problem was described by Kevin Sim et al 13 . Last year, Jiang Yuan et al use deep reinforc ement learning and constraint programming to solve 3D-BPP 14 .…”

Neural combinatorial optimization: from routing to integer programming

“…In recent years, deep learning has been broadly researched for optimization and applied to diverse COPs, e.g., routing [36][37][38], scheduling [12][13][14], knapsacking [39][40][41], bin packing [42][43][44]. Additional applications of deep learning to specific COPs, e.g., maximum cut, boolean satisfiability, graph coloring, can be found in survey papers [1,25,26].…”