SPHERE: An Evaluation Card for Human-AI Systems
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917982386520064 |
|---|---|
| author | Ma, Qianou Zhao, Dora Zhao, Xinran Si, Chenglei Yang, Chenyang Louie, Ryan Reiter, Ehud Yang, Diyi Wu, Tongshuang |
| author_facet | Ma, Qianou Zhao, Dora Zhao, Xinran Si, Chenglei Yang, Chenyang Louie, Ryan Reiter, Ehud Yang, Diyi Wu, Tongshuang |
| contents | In the era of Large Language Models (LLMs), establishing effective evaluation methods and standards for diverse human-AI interaction systems is increasingly challenging. To encourage more transparent documentation and facilitate discussion on human-AI system evaluation design options, we present an evaluation card SPHERE, which encompasses five key dimensions: 1) What is being evaluated?; 2) How is the evaluation conducted?; 3) Who is participating in the evaluation?; 4) When is evaluation conducted?; 5) How is evaluation validated? We conduct a review of 39 human-AI systems using SPHERE, outlining current evaluation practices and areas for improvement. We provide three recommendations for improving the validity and rigor of evaluation practices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07971 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SPHERE: An Evaluation Card for Human-AI Systems Ma, Qianou Zhao, Dora Zhao, Xinran Si, Chenglei Yang, Chenyang Louie, Ryan Reiter, Ehud Yang, Diyi Wu, Tongshuang Human-Computer Interaction Artificial Intelligence In the era of Large Language Models (LLMs), establishing effective evaluation methods and standards for diverse human-AI interaction systems is increasingly challenging. To encourage more transparent documentation and facilitate discussion on human-AI system evaluation design options, we present an evaluation card SPHERE, which encompasses five key dimensions: 1) What is being evaluated?; 2) How is the evaluation conducted?; 3) Who is participating in the evaluation?; 4) When is evaluation conducted?; 5) How is evaluation validated? We conduct a review of 39 human-AI systems using SPHERE, outlining current evaluation practices and areas for improvement. We provide three recommendations for improving the validity and rigor of evaluation practices. |
| title | SPHERE: An Evaluation Card for Human-AI Systems |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2504.07971 |