Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Huang, Sichao; Wang, Ziwei; Zhou, Jie; Lu, Jiwen

doi:10.1109/lra.2022.3222996

Cited by 10 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with conventional packing methods which respectively generate the object packing order randomly (Random), sorting the objects in descending order of area/ volume (B-Box Seq) [31], and employing the Biased Random Key Genetic Algorithm (BRKGA) [32] to optimal the object packing order. Moreover, the placement strategy based on a height map is applied to arranging the ordered objects in the boxes.…”

Section: Comparison With Conventional Packing Methodsmentioning

confidence: 99%

Learning to Solve 3-D Bin Packing Problem via Deep Reinforcement Learning and Constraint Programming

Jiang

Cao

Zhang

2023

IEEE Trans. Cybern.

View full text Add to dashboard Cite

The Bin Packing Problem (BPP) has attracted enthusiastic research interest recently, owing to widespread applications in logistics and warehousing environments. It is truly essential to optimize the bin packing to enable more objects to be packed into boxes. Object packing order and placement strategy are the two crucial optimization objectives of the BPP. However, existing optimization methods for BPP, such as the genetic algorithm (GA), emerge as the main issues in highly computational cost and relatively low accuracy, making it difficult to implement in realistic scenarios. To well relieve the research gaps, we present a novel optimization methodology of two-dimensional (2D)-BPP and three-dimensional (3D)-BPP for objects with regular shapes via deep reinforcement learning (DRL), maximizing the space utilization and minimizing the usage number of boxes. First, an end-to-end DRL neural network constructed by a modified Pointer Network consisting of an encoder, a decoder and an attention module is proposed to achieve the optimal object packing order. Second, conforming to the top-down operation mode, the placement strategy based on a height map is used to arrange the ordered objects in the boxes, preventing the objects from colliding with boxes and other objects in boxes. Third, the reward and loss functions are defined as the indicators of the compactness, pyramid, and usage number of boxes to conduct the training of the DRL neural network based on an on-policy actor-critic framework.Finally, a series of experiments are implemented to compare our method with conventional packing methods, from which we conclude that our method outperforms these packing methods in both packing accuracy and efficiency.

show abstract

Section: Comparison With Conventional Packing Methodsmentioning

confidence: 99%

Learning to Solve 3-D Bin Packing Problem via Deep Reinforcement Learning and Constraint Programming

Jiang

Cao

Zhang

2023

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…Recently, the bin packing problem has been solved through Deep Reinforcement Learning (DRL) [ 17 , 18 , 19 , 20 , 21 ]. Reinforcement Learning (RL) is a field of machine learning that entails a set of techniques for determining the optimum agent strategy and maximizing the reward of the agent [ 22 ].…”

Section: Literature Surveymentioning

confidence: 99%

BoxStacker: Deep Reinforcement Learning for 3D Bin Packing Problem in Virtual Environment of Logistics Systems

Murdivien,

2023

Sensors

View full text Add to dashboard Cite

Manufacturing systems need to be resilient and self-organizing to adapt to unexpected disruptions, such as product changes or rapid order, in supply chain changes while increasing the automation level of robotized logistics processes to cope with the lack of human experts. Deep Reinforcement Learning is a potential solution to solve more complex problems by introducing artificial neural networks in Reinforcement Learning. In this paper, a game engine was used for Deep Reinforcement Learning training, which allows visualization of view learning and result processes more intuitively than other tools, as well as a physical engine for a more realistic problem-solving environment. The present research demonstrates that a Deep Reinforcement Learning model can effectively address the real-time sequential 3D bin packing problem by utilizing a game engine to visualize the environment. The results indicate that this approach holds promise for tackling complex logistical challenges in dynamic settings.

show abstract

“…Robotic packing systems [21], [1], [34], [41], [13], [14] play a key role in warehouse automation with the benefits of reduced uptime, high throughput, and low accident rate compared with the labor-intensive approaches. The goal of robotic packing is to stow objects into constrained space such as shipping boxes.…”

Section: Introductionmentioning

confidence: 99%

Category-level Shape Estimation for Densely Cluttered Objects

Wu¹,

Wang²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

show abstract

Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Cited by 10 publications

References 37 publications

Learning to Solve 3-D Bin Packing Problem via Deep Reinforcement Learning and Constraint Programming

Learning to Solve 3-D Bin Packing Problem via Deep Reinforcement Learning and Constraint Programming

BoxStacker: Deep Reinforcement Learning for 3D Bin Packing Problem in Virtual Environment of Logistics Systems

Category-level Shape Estimation for Densely Cluttered Objects

Contact Info

Product

Resources

About