Playing with Words, Improving with Rewards: Training Language Models for Creative Association
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911723465736192 |
|---|---|
| author | Deshpande, Vijeta Shivagunde, Namrata Muckatira, Sherin Glaude, Hadrien Gronas, Mikhail Stevenson, Claire Beaty, Roger Rumshisky, Anna |
| author_facet | Deshpande, Vijeta Shivagunde, Namrata Muckatira, Sherin Glaude, Hadrien Gronas, Mikhail Stevenson, Claire Beaty, Roger Rumshisky, Anna |
| contents | Large Language Models (LLMs) are being applied to increasingly difficult problems and use cases. To navigate their vast solution spaces effectively, LLMs need to be creative. Yet the subjective nature of creativity and the limits of human judgment make training LLMs for creativity especially challenging. As a solution, we train LLMs on Codenames, a word-association game that exercises the two central axes of creativity, divergent and convergent thinking, while yielding objectively verifiable outcomes. This verifiability lets us bypass human judgment and train with Reinforcement Learning with Verifiable Rewards (RLVR). We train Qwen3-1.7B, 4B, and 8B models and evaluate them on ten creativity and four reasoning benchmarks. We find that the precision-diversity trade-off is scale-dependent: the 8B model prioritizes creativity over precision, while the 1.7B and 4B models gain reasoning precision at the cost of creativity. Concretely, the 8B model shows modest but consistent creativity gains (8 of 10 benchmarks) with only minor reasoning degradation, whereas the smaller models achieve substantial gains on reasoning tasks. Our study presents a scalable and effective solution to train LLMs for creativity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_27832 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Playing with Words, Improving with Rewards: Training Language Models for Creative Association Deshpande, Vijeta Shivagunde, Namrata Muckatira, Sherin Glaude, Hadrien Gronas, Mikhail Stevenson, Claire Beaty, Roger Rumshisky, Anna Computation and Language Large Language Models (LLMs) are being applied to increasingly difficult problems and use cases. To navigate their vast solution spaces effectively, LLMs need to be creative. Yet the subjective nature of creativity and the limits of human judgment make training LLMs for creativity especially challenging. As a solution, we train LLMs on Codenames, a word-association game that exercises the two central axes of creativity, divergent and convergent thinking, while yielding objectively verifiable outcomes. This verifiability lets us bypass human judgment and train with Reinforcement Learning with Verifiable Rewards (RLVR). We train Qwen3-1.7B, 4B, and 8B models and evaluate them on ten creativity and four reasoning benchmarks. We find that the precision-diversity trade-off is scale-dependent: the 8B model prioritizes creativity over precision, while the 1.7B and 4B models gain reasoning precision at the cost of creativity. Concretely, the 8B model shows modest but consistent creativity gains (8 of 10 benchmarks) with only minor reasoning degradation, whereas the smaller models achieve substantial gains on reasoning tasks. Our study presents a scalable and effective solution to train LLMs for creativity. |
| title | Playing with Words, Improving with Rewards: Training Language Models for Creative Association |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.27832 |