Deep hierarchical reinforcement learning for collaborative object transportation by heterogeneous agents

In the logistics and supply chain domain, coordinated efforts among agents play a pivotal role, particularly in the context of collaborative object transportation within a warehouse. This paper addresses the multifaceted challenge of multi -agent coordination in warehouse environments characterized by sparse reward structures, where the ability to communicate among agents may be limited or infeasible. Due to various constraints such as power limitations, weight capacity, or specialized abilities, the individual execution of this task by a single agent remains unattainable. Our study focuses on heterogeneous agents, where each agent possesses a distinct subset of skills and capabilities. Our research examines the emergence of cooperative behavior among groups of agents with the requisite skill sets, aiming to accomplish the task without explicit inter -agent communication or prior coordination. To encourage implicit agent coordination, we introduce a hierarchical approach integrating a global evaluation of abstract actions with curiosity -driven intrinsic learning. This approach is well -suited for real -world settings with scarce rewards. We evaluated its effectiveness in a warehouse domain, and the results show that our approach consistently achieves higher average returns, faster convergence, and improved exploration efficiency, highlighting its effectiveness in diverse scenarios.