POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918466521399296 |
|---|---|
| author | Chen, Yizhuo Liu, Xin Wang, Ruijie Li, Zheng Chen, Pei Yu, Changlong Yin, Qingyu Nigam, Priyanka Jiang, Meng Yin, Bing |
| author_facet | Chen, Yizhuo Liu, Xin Wang, Ruijie Li, Zheng Chen, Pei Yu, Changlong Yin, Qingyu Nigam, Priyanka Jiang, Meng Yin, Bing |
| contents | Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level personalization framework that separates the problem into two components connected by a natural-language interface: a shared inference model that distills heterogeneous user signals into a concise preference summary, and a shared generator that conditions on this summary to produce personalized responses. Both components are trained under a unified preference-optimization objective, with reinforcement learning handling the non-differentiable inference step. This objective decomposes into generator approximation error and summary informativeness, revealing how a single loss simultaneously drives accurate generation and informative summarization. Because the interface is natural language, learned summaries can be inferred once per user and reused across different generators -- including frozen, black-box commercial APIs. Across four personalization benchmarks, POPI generally improves personalization quality while reducing context overhead by up to an order of magnitude. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_17881 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | POPI: Personalizing LLMs via Optimized Natural Language Preference Inference Chen, Yizhuo Liu, Xin Wang, Ruijie Li, Zheng Chen, Pei Yu, Changlong Yin, Qingyu Nigam, Priyanka Jiang, Meng Yin, Bing Computation and Language Artificial Intelligence Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level personalization framework that separates the problem into two components connected by a natural-language interface: a shared inference model that distills heterogeneous user signals into a concise preference summary, and a shared generator that conditions on this summary to produce personalized responses. Both components are trained under a unified preference-optimization objective, with reinforcement learning handling the non-differentiable inference step. This objective decomposes into generator approximation error and summary informativeness, revealing how a single loss simultaneously drives accurate generation and informative summarization. Because the interface is natural language, learned summaries can be inferred once per user and reused across different generators -- including frozen, black-box commercial APIs. Across four personalization benchmarks, POPI generally improves personalization quality while reducing context overhead by up to an order of magnitude. |
| title | POPI: Personalizing LLMs via Optimized Natural Language Preference Inference |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.17881 |