Integrating Planning into Single-Turn Long-Form Text Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929533427384320 |
|---|---|
| author | Liang, Yi Wu, You Zhuang, Honglei Chen, Li Shen, Jiaming Jia, Yiling Qin, Zhen Sanghai, Sumit Wang, Xuanhui Yang, Carl Bendersky, Michael |
| author_facet | Liang, Yi Wu, You Zhuang, Honglei Chen, Li Shen, Jiaming Jia, Yiling Qin, Zhen Sanghai, Sumit Wang, Xuanhui Yang, Carl Bendersky, Michael |
| contents | Generating high-quality, in-depth textual documents, such as academic papers, news articles, Wikipedia entries, and books, remains a significant challenge for Large Language Models (LLMs). In this paper, we propose to use planning to generate long form content. To achieve our goal, we generate intermediate steps via an auxiliary task that teaches the LLM to plan, reason and structure before generating the final text. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. To overcome the scarcity of training data for these intermediate steps, we leverage LLMs to generate synthetic intermediate writing data such as outlines, key information and summaries from existing full articles. Our experiments demonstrate on two datasets from different domains, namely the scientific news dataset SciNews and Wikipedia datasets in KILT-Wiki and FreshWiki, that LLMs fine-tuned with the auxiliary task generate higher quality documents. We observed +2.5% improvement in ROUGE-Lsum, and a strong 3.60 overall win/loss ratio via human SxS evaluation, with clear wins in organization, relevance, and verifiability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06203 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Integrating Planning into Single-Turn Long-Form Text Generation Liang, Yi Wu, You Zhuang, Honglei Chen, Li Shen, Jiaming Jia, Yiling Qin, Zhen Sanghai, Sumit Wang, Xuanhui Yang, Carl Bendersky, Michael Computation and Language Artificial Intelligence Generating high-quality, in-depth textual documents, such as academic papers, news articles, Wikipedia entries, and books, remains a significant challenge for Large Language Models (LLMs). In this paper, we propose to use planning to generate long form content. To achieve our goal, we generate intermediate steps via an auxiliary task that teaches the LLM to plan, reason and structure before generating the final text. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. To overcome the scarcity of training data for these intermediate steps, we leverage LLMs to generate synthetic intermediate writing data such as outlines, key information and summaries from existing full articles. Our experiments demonstrate on two datasets from different domains, namely the scientific news dataset SciNews and Wikipedia datasets in KILT-Wiki and FreshWiki, that LLMs fine-tuned with the auxiliary task generate higher quality documents. We observed +2.5% improvement in ROUGE-Lsum, and a strong 3.60 overall win/loss ratio via human SxS evaluation, with clear wins in organization, relevance, and verifiability. |
| title | Integrating Planning into Single-Turn Long-Form Text Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.06203 |