Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.19038 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929522262147072 |
|---|---|
| author | Gimenez-Abalos, Victor Alvarez-Napagao, Sergio Tormos, Adrian Cortés, Ulises Vázquez-Salceda, Javier |
| author_facet | Gimenez-Abalos, Victor Alvarez-Napagao, Sergio Tormos, Adrian Cortés, Ulises Vázquez-Salceda, Javier |
| contents | Agents are a special kind of AI-based software in that they interact in complex environments and have increased potential for emergent behaviour. Explaining such emergent behaviour is key to deploying trustworthy AI, but the increasing complexity and opaque nature of many agent implementations makes this hard. In this work, we propose a Probabilistic Graphical Model along with a pipeline for designing such model -- by which the behaviour of an agent can be deliberated about -- and for computing a robust numerical value for the intentions the agent has at any moment. We contribute measurements that evaluate the interpretability and reliability of explanations provided, and enables explainability questions such as `what do you want to do now?' (e.g. deliver soup) `how do you plan to do it?' (e.g. returning a plan that considers its skills and the world), and `why would you take this action at this state?' (e.g. explaining how that furthers or hinders its own goals). This model can be constructed by taking partial observations of the agent's actions and world states, and we provide an iterative workflow for increasing the proposed measurements through better design and/or pointing out irrational agent behaviour. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Intention-aware policy graphs: answering what, how, and why in opaque agents Gimenez-Abalos, Victor Alvarez-Napagao, Sergio Tormos, Adrian Cortés, Ulises Vázquez-Salceda, Javier Artificial Intelligence Machine Learning Multiagent Systems Robotics 68T42 (Primary), 68T37, 68T05, 68Q87, 68T30, 68T40, 68M15 I.2; I.1; K.4; G.3 Agents are a special kind of AI-based software in that they interact in complex environments and have increased potential for emergent behaviour. Explaining such emergent behaviour is key to deploying trustworthy AI, but the increasing complexity and opaque nature of many agent implementations makes this hard. In this work, we propose a Probabilistic Graphical Model along with a pipeline for designing such model -- by which the behaviour of an agent can be deliberated about -- and for computing a robust numerical value for the intentions the agent has at any moment. We contribute measurements that evaluate the interpretability and reliability of explanations provided, and enables explainability questions such as `what do you want to do now?' (e.g. deliver soup) `how do you plan to do it?' (e.g. returning a plan that considers its skills and the world), and `why would you take this action at this state?' (e.g. explaining how that furthers or hinders its own goals). This model can be constructed by taking partial observations of the agent's actions and world states, and we provide an iterative workflow for increasing the proposed measurements through better design and/or pointing out irrational agent behaviour. |
| title | Intention-aware policy graphs: answering what, how, and why in opaque agents |
| topic | Artificial Intelligence Machine Learning Multiagent Systems Robotics 68T42 (Primary), 68T37, 68T05, 68Q87, 68T30, 68T40, 68M15 I.2; I.1; K.4; G.3 |
| url | https://arxiv.org/abs/2409.19038 |