Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.03304 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914963746979840 |
|---|---|
| author | Wang, Danqing Yang, Kevin Zhu, Hanlin Yang, Xiaomeng Cohen, Andrew Li, Lei Tian, Yuandong |
| author_facet | Wang, Danqing Yang, Kevin Zhu, Hanlin Yang, Xiaomeng Cohen, Andrew Li, Lei Tian, Yuandong |
| contents | Recent research has increasingly focused on evaluating large language models' (LLMs) alignment with diverse human values and preferences, particularly for open-ended tasks like story generation. Traditional evaluation metrics rely heavily on lexical similarity with human-written references, often showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. To address these challenges, we introduce PerSE, an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. PerSE enhances interpretability by providing detailed comments and fine-grained scoring, facilitating more personalized content generation. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers compared to GPT-4. It also outperforms GPT-4 by 46.01% in Kendall correlation on new domains, indicating its transferability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_03304 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Learning Personalized Alignment for Evaluating Open-ended Text Generation Wang, Danqing Yang, Kevin Zhu, Hanlin Yang, Xiaomeng Cohen, Andrew Li, Lei Tian, Yuandong Computation and Language Recent research has increasingly focused on evaluating large language models' (LLMs) alignment with diverse human values and preferences, particularly for open-ended tasks like story generation. Traditional evaluation metrics rely heavily on lexical similarity with human-written references, often showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. To address these challenges, we introduce PerSE, an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. PerSE enhances interpretability by providing detailed comments and fine-grained scoring, facilitating more personalized content generation. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers compared to GPT-4. It also outperforms GPT-4 by 46.01% in Kendall correlation on new domains, indicating its transferability. |
| title | Learning Personalized Alignment for Evaluating Open-ended Text Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2310.03304 |