Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910776896258048 |
|---|---|
| author | Salemi, Alireza Li, Cheng Zhang, Mingyang Mei, Qiaozhu Kong, Weize Chen, Tao Li, Zhuowan Bendersky, Michael Zamani, Hamed |
| author_facet | Salemi, Alireza Li, Cheng Zhang, Mingyang Mei, Qiaozhu Kong, Weize Chen, Tao Li, Zhuowan Bendersky, Michael Zamani, Hamed |
| contents | Personalized text generation requires a unique ability of large language models (LLMs) to learn from context that they often do not encounter during their standard training. One way to encourage LLMs to better use personalized context for generating outputs that better align with the user's expectations is to instruct them to reason over the user's past preferences, background knowledge, or writing style. To achieve this, we propose Reasoning-Enhanced Self-Training for Personalized Text Generation (REST-PG), a framework that trains LLMs to reason over personal data during response generation. REST-PG first generates reasoning paths to train the LLM's reasoning abilities and then employs Expectation-Maximization Reinforced Self-Training to iteratively train the LLM based on its own high-reward outputs. We evaluate REST-PG on the LongLaMP benchmark, consisting of four diverse personalized long-form text generation tasks. Our experiments demonstrate that REST-PG achieves significant improvements over state-of-the-art baselines, with an average relative performance gain of 14.5% on the benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04167 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation Salemi, Alireza Li, Cheng Zhang, Mingyang Mei, Qiaozhu Kong, Weize Chen, Tao Li, Zhuowan Bendersky, Michael Zamani, Hamed Computation and Language Artificial Intelligence Information Retrieval Personalized text generation requires a unique ability of large language models (LLMs) to learn from context that they often do not encounter during their standard training. One way to encourage LLMs to better use personalized context for generating outputs that better align with the user's expectations is to instruct them to reason over the user's past preferences, background knowledge, or writing style. To achieve this, we propose Reasoning-Enhanced Self-Training for Personalized Text Generation (REST-PG), a framework that trains LLMs to reason over personal data during response generation. REST-PG first generates reasoning paths to train the LLM's reasoning abilities and then employs Expectation-Maximization Reinforced Self-Training to iteratively train the LLM based on its own high-reward outputs. We evaluate REST-PG on the LongLaMP benchmark, consisting of four diverse personalized long-form text generation tasks. Our experiments demonstrate that REST-PG achieves significant improvements over state-of-the-art baselines, with an average relative performance gain of 14.5% on the benchmark. |
| title | Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation |
| topic | Computation and Language Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2501.04167 |