XEQ Scale for Evaluating XAI Experience Quality
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917894320816128 |
|---|---|
| author | Wijekoon, Anjana Wiratunga, Nirmalie Corsar, David Martin, Kyle Nkisi-Orji, Ikechukwu Díaz-Agudo, Belen Bridge, Derek |
| author_facet | Wijekoon, Anjana Wiratunga, Nirmalie Corsar, David Martin, Kyle Nkisi-Orji, Ikechukwu Díaz-Agudo, Belen Bridge, Derek |
| contents | Explainable Artificial Intelligence (XAI) aims to improve the transparency of autonomous decision-making through explanations. Recent literature has emphasised users' need for holistic "multi-shot" explanations and personalised engagement with XAI systems. We refer to this user-centred interaction as an XAI Experience. Despite advances in creating XAI experiences, evaluating them in a user-centred manner has remained challenging. In response, we developed the XAI Experience Quality (XEQ) Scale. XEQ quantifies the quality of experiences across four dimensions: learning, utility, fulfilment and engagement. These contributions extend the state-of-the-art of XAI evaluation, moving beyond the one-dimensional metrics frequently developed to assess single-shot explanations. This paper presents the XEQ scale development and validation process, including content validation with XAI experts, and discriminant and construct validation through a large-scale pilot study. Our pilot study results offer strong evidence that establishes the XEQ Scale as a comprehensive framework for evaluating user-centred XAI experiences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10662 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | XEQ Scale for Evaluating XAI Experience Quality Wijekoon, Anjana Wiratunga, Nirmalie Corsar, David Martin, Kyle Nkisi-Orji, Ikechukwu Díaz-Agudo, Belen Bridge, Derek Artificial Intelligence Human-Computer Interaction Explainable Artificial Intelligence (XAI) aims to improve the transparency of autonomous decision-making through explanations. Recent literature has emphasised users' need for holistic "multi-shot" explanations and personalised engagement with XAI systems. We refer to this user-centred interaction as an XAI Experience. Despite advances in creating XAI experiences, evaluating them in a user-centred manner has remained challenging. In response, we developed the XAI Experience Quality (XEQ) Scale. XEQ quantifies the quality of experiences across four dimensions: learning, utility, fulfilment and engagement. These contributions extend the state-of-the-art of XAI evaluation, moving beyond the one-dimensional metrics frequently developed to assess single-shot explanations. This paper presents the XEQ scale development and validation process, including content validation with XAI experts, and discriminant and construct validation through a large-scale pilot study. Our pilot study results offer strong evidence that establishes the XEQ Scale as a comprehensive framework for evaluating user-centred XAI experiences. |
| title | XEQ Scale for Evaluating XAI Experience Quality |
| topic | Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2407.10662 |