Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.16956 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917026895757312 |
|---|---|
| author | Towers, Mark Du, Yali Freeman, Christopher Norman, Timothy J. |
| author_facet | Towers, Mark Du, Yali Freeman, Christopher Norman, Timothy J. |
| contents | Debugging is a core application of explainable reinforcement learning (XRL) algorithms; however, limited comparative evaluations have been conducted to understand their relative performance. We propose a novel evaluation methodology to test whether users can identify an agent's goal from an explanation of its decision-making. Utilising the Atari's Ms. Pacman environment and four XRL algorithms, we find that only one achieved greater than random accuracy for the tested goals and that users were generally overconfident in their selections. Further, we find that users' self-reported ease of identification and understanding for every explanation did not correlate with their accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_16956 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Comparative User Evaluation of XRL Explanations using Goal Identification Towers, Mark Du, Yali Freeman, Christopher Norman, Timothy J. Artificial Intelligence Debugging is a core application of explainable reinforcement learning (XRL) algorithms; however, limited comparative evaluations have been conducted to understand their relative performance. We propose a novel evaluation methodology to test whether users can identify an agent's goal from an explanation of its decision-making. Utilising the Atari's Ms. Pacman environment and four XRL algorithms, we find that only one achieved greater than random accuracy for the tested goals and that users were generally overconfident in their selections. Further, we find that users' self-reported ease of identification and understanding for every explanation did not correlate with their accuracy. |
| title | A Comparative User Evaluation of XRL Explanations using Goal Identification |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.16956 |