Emergence in Multi-Agent Systems: A Safety Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913462346579968 |
|---|---|
| author | Altmann, Philipp Schönberger, Julian Illium, Steffen Zorn, Maximilian Ritz, Fabian Haider, Tom Burton, Simon Gabor, Thomas |
| author_facet | Altmann, Philipp Schönberger, Julian Illium, Steffen Zorn, Maximilian Ritz, Fabian Haider, Tom Burton, Simon Gabor, Thomas |
| contents | Emergent effects can arise in multi-agent systems (MAS) where execution is decentralized and reliant on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these effects, we identify misalignments between the global inherent specification (the true specification) and its local approximation (such as the configuration of different reward components or observations). Using established safety terminology, we develop a framework to understand these emergent effects. To showcase the resulting implications, we use two broadly configurable exemplary gridworld scenarios, where insufficient specification leads to unintended behavior deviations when derived independently. Recognizing that a global adaptation might not always be feasible, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving the system's alignment and reducing the risk of emergent failures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04514 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Emergence in Multi-Agent Systems: A Safety Perspective Altmann, Philipp Schönberger, Julian Illium, Steffen Zorn, Maximilian Ritz, Fabian Haider, Tom Burton, Simon Gabor, Thomas Multiagent Systems Emergent effects can arise in multi-agent systems (MAS) where execution is decentralized and reliant on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these effects, we identify misalignments between the global inherent specification (the true specification) and its local approximation (such as the configuration of different reward components or observations). Using established safety terminology, we develop a framework to understand these emergent effects. To showcase the resulting implications, we use two broadly configurable exemplary gridworld scenarios, where insufficient specification leads to unintended behavior deviations when derived independently. Recognizing that a global adaptation might not always be feasible, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving the system's alignment and reducing the risk of emergent failures. |
| title | Emergence in Multi-Agent Systems: A Safety Perspective |
| topic | Multiagent Systems |
| url | https://arxiv.org/abs/2408.04514 |