TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910368575520768 |
|---|---|
| author | Jiang, Pengcheng Xiao, Cao Wang, Zifeng Bhatia, Parminder Sun, Jimeng Han, Jiawei |
| author_facet | Jiang, Pengcheng Xiao, Cao Wang, Zifeng Bhatia, Parminder Sun, Jimeng Han, Jiawei |
| contents | The advent of large language models (LLMs) has significantly advanced natural language processing tasks like text summarization. However, their large size and computational demands, coupled with privacy concerns in data transmission, limit their use in resource-constrained and privacy-centric settings. To overcome this, we introduce TriSum, a framework for distilling LLMs' text summarization abilities into a compact, local model. Initially, LLMs extract a set of aspect-triple rationales and summaries, which are refined using a dual-scoring method for quality. Next, a smaller local model is trained with these tasks, employing a curriculum learning strategy that evolves from simple to complex tasks. Our method enhances local model performance on various benchmarks (CNN/DailyMail, XSum, and ClinicalTrial), outperforming baselines by 4.5%, 8.5%, and 7.4%, respectively. It also improves interpretability by providing insights into the summarization rationale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10351 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale Jiang, Pengcheng Xiao, Cao Wang, Zifeng Bhatia, Parminder Sun, Jimeng Han, Jiawei Computation and Language The advent of large language models (LLMs) has significantly advanced natural language processing tasks like text summarization. However, their large size and computational demands, coupled with privacy concerns in data transmission, limit their use in resource-constrained and privacy-centric settings. To overcome this, we introduce TriSum, a framework for distilling LLMs' text summarization abilities into a compact, local model. Initially, LLMs extract a set of aspect-triple rationales and summaries, which are refined using a dual-scoring method for quality. Next, a smaller local model is trained with these tasks, employing a curriculum learning strategy that evolves from simple to complex tasks. Our method enhances local model performance on various benchmarks (CNN/DailyMail, XSum, and ClinicalTrial), outperforming baselines by 4.5%, 8.5%, and 7.4%, respectively. It also improves interpretability by providing insights into the summarization rationale. |
| title | TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.10351 |