Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2407.13248 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916424320024576 |
|---|---|
| author | Tian, Yufei Huang, Tenghao Liu, Miri Jiang, Derek Spangher, Alexander Chen, Muhao May, Jonathan Peng, Nanyun |
| author_facet | Tian, Yufei Huang, Tenghao Liu, Miri Jiang, Derek Spangher, Alexander Chen, Muhao May, Jonathan Peng, Nanyun |
| contents | This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression. We introduce a novel computational framework to analyze narratives through three discourse-level aspects: i) story arcs, ii) turning points, and iii) affective dimensions, including arousal and valence. By leveraging expert and automatic annotations, we uncover significant discrepancies between the LLM- and human- written stories. While human-written stories are suspenseful, arousing, and diverse in narrative structures, LLM stories are homogeneously positive and lack tension. Next, we measure narrative reasoning skills as a precursor to generative capacities, concluding that most LLMs fall short of human abilities in discourse understanding. Finally, we show that explicit integration of aforementioned discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling in terms of diversity, suspense, and arousal. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13248 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are Large Language Models Capable of Generating Human-Level Narratives? Tian, Yufei Huang, Tenghao Liu, Miri Jiang, Derek Spangher, Alexander Chen, Muhao May, Jonathan Peng, Nanyun Computation and Language This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression. We introduce a novel computational framework to analyze narratives through three discourse-level aspects: i) story arcs, ii) turning points, and iii) affective dimensions, including arousal and valence. By leveraging expert and automatic annotations, we uncover significant discrepancies between the LLM- and human- written stories. While human-written stories are suspenseful, arousing, and diverse in narrative structures, LLM stories are homogeneously positive and lack tension. Next, we measure narrative reasoning skills as a precursor to generative capacities, concluding that most LLMs fall short of human abilities in discourse understanding. Finally, we show that explicit integration of aforementioned discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling in terms of diversity, suspense, and arousal. |
| title | Are Large Language Models Capable of Generating Human-Level Narratives? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.13248 |