Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.14773 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917413283430400 |
|---|---|
| author | Su, Hang Liu, Zequn Hu, Chen Lu, Xuesong Xia, Yingce Liu, Zhen |
| author_facet | Su, Hang Liu, Zequn Hu, Chen Lu, Xuesong Xia, Yingce Liu, Zhen |
| contents | While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradigms predominantly rely on lexical-level similarity or manual heuristics, often lacking sufficient data-driven validation. We address this by mining Community-Individual Preference Divergence (CIPD), where individual choices override consensus, to distill six key personalization factors as evaluative dimensions. Accordingly, we introduce CoPA, a benchmark with 1,985 user profiles for fine-grained, factor-level assessment. By quantifying the alignment between model outputs and user-specific cognitive preferences inferred from interaction patterns, CoPA provides a more comprehensive and discriminative standard for evaluating personalized QA than generic metrics. The code is available at https://github.com/bjzgcai/CoPA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14773 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors Su, Hang Liu, Zequn Hu, Chen Lu, Xuesong Xia, Yingce Liu, Zhen Computation and Language While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradigms predominantly rely on lexical-level similarity or manual heuristics, often lacking sufficient data-driven validation. We address this by mining Community-Individual Preference Divergence (CIPD), where individual choices override consensus, to distill six key personalization factors as evaluative dimensions. Accordingly, we introduce CoPA, a benchmark with 1,985 user profiles for fine-grained, factor-level assessment. By quantifying the alignment between model outputs and user-specific cognitive preferences inferred from interaction patterns, CoPA provides a more comprehensive and discriminative standard for evaluating personalized QA than generic metrics. The code is available at https://github.com/bjzgcai/CoPA. |
| title | CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.14773 |