Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909632279085056 |
|---|---|
| author | Mok, Jisoo Kim, Ik-hwan Park, Sangkwon Yoon, Sungroh |
| author_facet | Mok, Jisoo Kim, Ik-hwan Park, Sangkwon Yoon, Sungroh |
| contents | Personalized AI assistants, a hallmark of the human-like capabilities of Large Language Models (LLMs), are a challenging application that intertwines multiple problems in LLM research. Despite the growing interest in the development of personalized assistants, the lack of an open-source conversational dataset tailored for personalization remains a significant obstacle for researchers in the field. To address this research gap, we introduce HiCUPID, a new benchmark to probe and unleash the potential of LLMs to deliver personalized responses. Alongside a conversational dataset, HiCUPID provides a Llama-3.2-based automated evaluation model whose assessment closely mirrors human preferences. We release our dataset, evaluation model, and code at https://github.com/12kimih/HiCUPID. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01262 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis Mok, Jisoo Kim, Ik-hwan Park, Sangkwon Yoon, Sungroh Computation and Language Personalized AI assistants, a hallmark of the human-like capabilities of Large Language Models (LLMs), are a challenging application that intertwines multiple problems in LLM research. Despite the growing interest in the development of personalized assistants, the lack of an open-source conversational dataset tailored for personalization remains a significant obstacle for researchers in the field. To address this research gap, we introduce HiCUPID, a new benchmark to probe and unleash the potential of LLMs to deliver personalized responses. Alongside a conversational dataset, HiCUPID provides a Llama-3.2-based automated evaluation model whose assessment closely mirrors human preferences. We release our dataset, evaluation model, and code at https://github.com/12kimih/HiCUPID. |
| title | Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.01262 |