Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.09609 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912824118214656 |
|---|---|
| author | Cao, Qian Liu, Yahui Bi, Wei Zhao, Yi Song, Ruihua Wang, Xiting Tang, Ruiming Zhou, Guorui Li, Han |
| author_facet | Cao, Qian Liu, Yahui Bi, Wei Zhao, Yi Song, Ruihua Wang, Xiting Tang, Ruiming Zhou, Guorui Li, Han |
| contents | Reinforcement learning (RL)-based enhancement of large language models (LLMs) often leads to reduced output diversity, undermining their utility in open-ended tasks like creative writing. Current methods lack explicit mechanisms for guiding diverse exploration and instead prioritize optimization efficiency and performance over diversity. This paper proposes an RL framework structured around a semi-structured long Chain-of-Thought (CoT), in which the generation process is decomposed into explicitly planned intermediate steps. We introduce a Diverse Planning Branching method that strategically introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. Experimental results on creative writing benchmarks demonstrate that our approach significantly improves output diversity without compromising generation quality, consistently outperforming existing baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_09609 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing Cao, Qian Liu, Yahui Bi, Wei Zhao, Yi Song, Ruihua Wang, Xiting Tang, Ruiming Zhou, Guorui Li, Han Computation and Language Artificial Intelligence Reinforcement learning (RL)-based enhancement of large language models (LLMs) often leads to reduced output diversity, undermining their utility in open-ended tasks like creative writing. Current methods lack explicit mechanisms for guiding diverse exploration and instead prioritize optimization efficiency and performance over diversity. This paper proposes an RL framework structured around a semi-structured long Chain-of-Thought (CoT), in which the generation process is decomposed into explicitly planned intermediate steps. We introduce a Diverse Planning Branching method that strategically introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. Experimental results on creative writing benchmarks demonstrate that our approach significantly improves output diversity without compromising generation quality, consistently outperforming existing baselines. |
| title | DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.09609 |