Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916374983475200 |
|---|---|
| author | Krnjaic, Aleksandar Steleac, Raul D. Thomas, Jonathan D. Papoudakis, Georgios Schäfer, Lukas To, Andrew Wing Keung Lao, Kuan-Ho Cubuktepe, Murat Haley, Matthew Börsting, Peter Albrecht, Stefano V. |
| author_facet | Krnjaic, Aleksandar Steleac, Raul D. Thomas, Jonathan D. Papoudakis, Georgios Schäfer, Lukas To, Andrew Wing Keung Lao, Kuan-Ho Cubuktepe, Murat Haley, Matthew Börsting, Peter Albrecht, Stefano V. |
| contents | We consider a warehouse in which dozens of mobile robots and human pickers work together to collect and deliver items within the warehouse. The fundamental problem we tackle, called the order-picking problem, is how these worker agents must coordinate their movement and actions in the warehouse to maximise performance in this task. Established industry methods using heuristic approaches require large engineering efforts to optimise for innately variable warehouse configurations. In contrast, multi-agent reinforcement learning (MARL) can be flexibly applied to diverse warehouse configurations (e.g. size, layout, number/types of workers, item replenishment frequency), and different types of order-picking paradigms (e.g. Goods-to-Person and Person-to-Goods), as the agents can learn how to cooperate optimally through experience. We develop hierarchical MARL algorithms in which a manager agent assigns goals to worker agents, and the policies of the manager and workers are co-trained toward maximising a global objective (e.g. pick rate). Our hierarchical algorithms achieve significant gains in sample efficiency over baseline MARL algorithms and overall pick rates over multiple established industry heuristics in a diverse set of warehouse configurations and different order-picking paradigms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_11498 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers Krnjaic, Aleksandar Steleac, Raul D. Thomas, Jonathan D. Papoudakis, Georgios Schäfer, Lukas To, Andrew Wing Keung Lao, Kuan-Ho Cubuktepe, Murat Haley, Matthew Börsting, Peter Albrecht, Stefano V. Machine Learning Artificial Intelligence Multiagent Systems Robotics We consider a warehouse in which dozens of mobile robots and human pickers work together to collect and deliver items within the warehouse. The fundamental problem we tackle, called the order-picking problem, is how these worker agents must coordinate their movement and actions in the warehouse to maximise performance in this task. Established industry methods using heuristic approaches require large engineering efforts to optimise for innately variable warehouse configurations. In contrast, multi-agent reinforcement learning (MARL) can be flexibly applied to diverse warehouse configurations (e.g. size, layout, number/types of workers, item replenishment frequency), and different types of order-picking paradigms (e.g. Goods-to-Person and Person-to-Goods), as the agents can learn how to cooperate optimally through experience. We develop hierarchical MARL algorithms in which a manager agent assigns goals to worker agents, and the policies of the manager and workers are co-trained toward maximising a global objective (e.g. pick rate). Our hierarchical algorithms achieve significant gains in sample efficiency over baseline MARL algorithms and overall pick rates over multiple established industry heuristics in a diverse set of warehouse configurations and different order-picking paradigms. |
| title | Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers |
| topic | Machine Learning Artificial Intelligence Multiagent Systems Robotics |
| url | https://arxiv.org/abs/2212.11498 |