Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.09310 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911261907746816 |
|---|---|
| author | Chung, John Joon Young Padmakumar, Vishakh Roemmele, Melissa Wang, Yi Sun, Yuqian Wang, Tiffany Almeda, Shm Garanganao Halperin, Brett A. Lu, Yuwen Kreminski, Max |
| author_facet | Chung, John Joon Young Padmakumar, Vishakh Roemmele, Melissa Wang, Yi Sun, Yuqian Wang, Tiffany Almeda, Shm Garanganao Halperin, Brett A. Lu, Yuwen Kreminski, Max |
| contents | People have different creative writing preferences, and large language models (LLMs) for these tasks can benefit from adapting to each user's preferences. However, these models are often trained over a dataset that considers varying personal tastes as a monolith. To facilitate developing personalized creative writing LLMs, we introduce LiteraryTaste, a dataset of reading preferences from 60 people, where each person: 1) self-reported their reading habits and tastes (stated preference), and 2) annotated their preferences over 100 pairs of short creative writing texts (revealed preference). With our dataset, we found that: 1) people diverge on creative writing preferences, 2) finetuning a transformer encoder could achieve 75.8% and 67.7% accuracy when modeling personal and collective revealed preferences, and 3) stated preferences had limited utility in modeling revealed preferences. With an LLM-driven interpretability pipeline, we analyzed how people's preferences vary. We hope our work serves as a cornerstone for personalizing creative writing technologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_09310 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LiteraryTaste: A Preference Dataset for Creative Writing Personalization Chung, John Joon Young Padmakumar, Vishakh Roemmele, Melissa Wang, Yi Sun, Yuqian Wang, Tiffany Almeda, Shm Garanganao Halperin, Brett A. Lu, Yuwen Kreminski, Max Computation and Language Human-Computer Interaction People have different creative writing preferences, and large language models (LLMs) for these tasks can benefit from adapting to each user's preferences. However, these models are often trained over a dataset that considers varying personal tastes as a monolith. To facilitate developing personalized creative writing LLMs, we introduce LiteraryTaste, a dataset of reading preferences from 60 people, where each person: 1) self-reported their reading habits and tastes (stated preference), and 2) annotated their preferences over 100 pairs of short creative writing texts (revealed preference). With our dataset, we found that: 1) people diverge on creative writing preferences, 2) finetuning a transformer encoder could achieve 75.8% and 67.7% accuracy when modeling personal and collective revealed preferences, and 3) stated preferences had limited utility in modeling revealed preferences. With an LLM-driven interpretability pipeline, we analyzed how people's preferences vary. We hope our work serves as a cornerstone for personalizing creative writing technologies. |
| title | LiteraryTaste: A Preference Dataset for Creative Writing Personalization |
| topic | Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2511.09310 |