Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.12109 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912725740814336 |
|---|---|
| author | Bu, Hyungjune Jung, Chanjoo Kang, Minjae Kim, Jaehyung |
| author_facet | Bu, Hyungjune Jung, Chanjoo Kang, Minjae Kim, Jaehyung |
| contents | As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach applied after performing parameter-efficient fine-tuning (PEFT) on user-specific data. Our core idea is to leverage reward-guided decoding specifically for personalization by maximizing each user's implicit reward signal. We evaluate CoPe across five open-ended personalized text generation tasks. Our empirical results demonstrate that CoPe achieves strong performance, improving personalization by an average of 10.57% in ROUGE-L, without relying on external reward models or additional training procedures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12109 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Personalized LLM Decoding via Contrasting Personal Preference Bu, Hyungjune Jung, Chanjoo Kang, Minjae Kim, Jaehyung Computation and Language Artificial Intelligence As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach applied after performing parameter-efficient fine-tuning (PEFT) on user-specific data. Our core idea is to leverage reward-guided decoding specifically for personalization by maximizing each user's implicit reward signal. We evaluate CoPe across five open-ended personalized text generation tasks. Our empirical results demonstrate that CoPe achieves strong performance, improving personalization by an average of 10.57% in ROUGE-L, without relying on external reward models or additional training procedures. |
| title | Personalized LLM Decoding via Contrasting Personal Preference |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.12109 |