EvAlignUX: Advancing UX Evaluation through LLM-Supported Metrics Exploration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913930934222848 |
|---|---|
| author | Zheng, Qingxiao Chen, Minrui Sharma, Pranav Tang, Yiliu Oswal, Mehul Liu, Yiren Huang, Yun |
| author_facet | Zheng, Qingxiao Chen, Minrui Sharma, Pranav Tang, Yiliu Oswal, Mehul Liu, Yiren Huang, Yun |
| contents | Evaluating UX in the context of AI's complexity, unpredictability, and generative nature presents unique challenges. How can we support HCI researchers to create comprehensive UX evaluation plans? In this paper, we introduce EvAlignUX, a system powered by large language models and grounded in scientific literature, designed to help HCI researchers explore evaluation metrics and their relationship to research outcomes. A user study with 19 HCI scholars showed that EvAlignUX improved the perceived quality and confidence in UX evaluation plans while prompting deeper consideration of research impact and risks. The system enhanced participants' thought processes, leading to the creation of a ``UX Question Bank'' to guide UX evaluation development. Findings also highlight how researchers' backgrounds influence their inspiration and concerns about AI over-reliance, pointing to future research on AI's role in fostering critical thinking. In a world where experience defines impact, we discuss the importance of shifting UX evaluation from a ``method-centric'' to a ``mindset-centric'' approach as the key to meaningful and lasting design evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15471 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EvAlignUX: Advancing UX Evaluation through LLM-Supported Metrics Exploration Zheng, Qingxiao Chen, Minrui Sharma, Pranav Tang, Yiliu Oswal, Mehul Liu, Yiren Huang, Yun Human-Computer Interaction Evaluating UX in the context of AI's complexity, unpredictability, and generative nature presents unique challenges. How can we support HCI researchers to create comprehensive UX evaluation plans? In this paper, we introduce EvAlignUX, a system powered by large language models and grounded in scientific literature, designed to help HCI researchers explore evaluation metrics and their relationship to research outcomes. A user study with 19 HCI scholars showed that EvAlignUX improved the perceived quality and confidence in UX evaluation plans while prompting deeper consideration of research impact and risks. The system enhanced participants' thought processes, leading to the creation of a ``UX Question Bank'' to guide UX evaluation development. Findings also highlight how researchers' backgrounds influence their inspiration and concerns about AI over-reliance, pointing to future research on AI's role in fostering critical thinking. In a world where experience defines impact, we discuss the importance of shifting UX evaluation from a ``method-centric'' to a ``mindset-centric'' approach as the key to meaningful and lasting design evaluation. |
| title | EvAlignUX: Advancing UX Evaluation through LLM-Supported Metrics Exploration |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2409.15471 |