Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Liu, Shanqi; Wen, Licheng; Cui, Jinhao; Yang, Xin; Cao, Junjie; Liu, Yong

doi:10.1109/iros51168.2021.9636224

Cited by 4 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The simulations demonstrated the effectiveness of the CI in optimizing time, efficiency, and safety in an automated warehouse system. In the context of multi-robot tasks in warehouses, Liu et al [30] addressed multi-agent path finding (MAPF) in formation. The authors proposed a decentralized partially observable RL algorithm that used a hierarchical structure to decompose the task into unrelated sub-tasks.…”

Section: Applications Of Rl With Simulation In Warehouse Operationsmentioning

confidence: 99%

“…The reasons are obviously the increased complexity of handling the inter-software communication and the potentially richer environment from which the agent needs to learn, as exemplified in the present work with FlexSim. Drakaki and Tzionas [16] Kono et al [17] Li et al [18] Sartoretti et al [19] Li et al [20] Barat et al [21] Sun and Li [22] Xiao et al [23] Yang et al [24] Ushida et al [25] Shen et al [26] Newaz and Alam [27] CoppeliaSim Peyas et al [28] Ha et al [29] Liu et al [30] Ushida et al [31] Lee and Jeong [32] Tang et al [33] Li et al [34] CloudSim…”

Section: Advancements In Rl With Simulation For Warehouse Operationsmentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Simulation and Reinforcement Learning Algorithm for Enhancing Efficiency in Warehouse Operations

Leon,

Li,

Martin

et al. 2023

Algorithms

View full text Add to dashboard Cite

The use of simulation and reinforcement learning can be viewed as a flexible approach to aid managerial decision-making, particularly in the face of growing complexity in manufacturing and logistic systems. Efficient supply chains heavily rely on steamlined warehouse operations, and therefore, having a well-informed storage location assignment policy is crucial for their improvement. The traditional methods found in the literature for tackling the storage location assignment problem have certain drawbacks, including the omission of stochastic process variability or the neglect of interaction between various warehouse workers. In this context, we explore the possibilities of combining simulation with reinforcement learning to develop effective mechanisms that allow for the quick acquisition of information about a complex environment, the processing of that information, and then the decision-making about the best storage location assignment. In order to test these concepts, we will make use of the FlexSim commercial simulator.

show abstract

Section: Applications Of Rl With Simulation In Warehouse Operationsmentioning

confidence: 99%

Section: Advancements In Rl With Simulation For Warehouse Operationsmentioning

confidence: 99%

A Hybrid Simulation and Reinforcement Learning Algorithm for Enhancing Efficiency in Warehouse Operations

Leon,

Li,

Martin

et al. 2023

Algorithms

View full text Add to dashboard Cite

show abstract

“…Many real-world tasks like bimanual manipulation [11], autonomous driving [13] and swarms [8] require the cooperation of multiple agents. Especially, in many scenarios agents are expected to choose the optimal actions simultaneously to complete the common goal, such as the bimanual lifting task requiring dual arms to lift the object simultaneously [1,2].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning

Liu¹,

Hu²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Real-world cooperation often requires intensive coordination among agents simultaneously. This task has been extensively studied within the framework of cooperative multi-agent reinforcement learning (MARL), and value decomposition methods are among those cuttingedge solutions. However, traditional methods that learn the value function as a monotonic mixing of per-agent utilities cannot solve the tasks with non-monotonic returns. This hinders their application in generic scenarios. Recent methods tackle this problem from the perspective of implicit credit assignment by learning value functions with complete expressiveness or using additional structures to improve cooperation. However, they are either difficult to learn due to large joint action spaces or insufficient to capture the complicated interactions among agents which are essential to solving tasks with non-monotonic returns. Moreover, applications in real-world scenarios usually require policies to be interpretable, but interpretability is limited in the implicit credit assignment methods. To address these problems, we propose a novel explicit credit assignment method to address the non-monotonic problem. Our method, Adaptive Value decomposition with Greedy Marginal contribution (AVGM), is based on an adaptive value decomposition that learns the cooperative value of a group of dynamically changing agents. We first illustrate that the proposed value decomposition can consider the complicated interactions among agents and is feasible to learn in large-scale scenarios. Then, our method uses a greedy marginal contribution computed from the value decomposition as an individual credit to incentivize agents to learn the optimal cooperative policy. We further extend the module with an action encoder to guarantee the linear time complexity for computing the greedy marginal contribution. Experimental results demonstrate that our method achieves significant performance improvements

show abstract

Expert demonstrations guide reward decomposition for multi-agent cooperation

Weiwei,

Wei,

Shanqi

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Cited by 4 publications

References 16 publications

A Hybrid Simulation and Reinforcement Learning Algorithm for Enhancing Efficiency in Warehouse Operations

A Hybrid Simulation and Reinforcement Learning Algorithm for Enhancing Efficiency in Warehouse Operations

Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning

Expert demonstrations guide reward decomposition for multi-agent cooperation

Contact Info

Product

Resources

About