Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.17640 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909463598858240 |
|---|---|
| author | Tong, Xindi Zhu, Yujin Fan, Shijian Xu, Liang |
| author_facet | Tong, Xindi Zhu, Yujin Fan, Shijian Xu, Liang |
| contents | Long text summarization, gradually being essential for efficiently processing large volumes of information, stays challenging for Large Language Models (LLMs) such as GPT and LLaMA families because of the insufficient open-sourced training datasets and the high requirement of contextual details dealing. To address the issue, we design a novel zero-shot transfer learning framework, abbreviated as T3, to iteratively training a baseline LLM on an assistant task for the target task, where the former should own richer data resources and share structural or semantic similarity with the latter. In practice, T3 is approached to deal with the long text summarization task by utilizing question answering as the assistant task, and further validated its effectiveness on the BBC summary, NarraSum, FairytaleQA, and NLQuAD datasets, with up to nearly 14% improvement in ROUGE, 35% improvement in BLEU, and 16% improvement in Factscore compared to three baseline LLMs, demonstrating its potential for more assistant-target task combinations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_17640 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | T3: A Novel Zero-shot Transfer Learning Framework Iteratively Training on an Assistant Task for a Target Task Tong, Xindi Zhu, Yujin Fan, Shijian Xu, Liang Computation and Language Artificial Intelligence Long text summarization, gradually being essential for efficiently processing large volumes of information, stays challenging for Large Language Models (LLMs) such as GPT and LLaMA families because of the insufficient open-sourced training datasets and the high requirement of contextual details dealing. To address the issue, we design a novel zero-shot transfer learning framework, abbreviated as T3, to iteratively training a baseline LLM on an assistant task for the target task, where the former should own richer data resources and share structural or semantic similarity with the latter. In practice, T3 is approached to deal with the long text summarization task by utilizing question answering as the assistant task, and further validated its effectiveness on the BBC summary, NarraSum, FairytaleQA, and NLQuAD datasets, with up to nearly 14% improvement in ROUGE, 35% improvement in BLEU, and 16% improvement in Factscore compared to three baseline LLMs, demonstrating its potential for more assistant-target task combinations. |
| title | T3: A Novel Zero-shot Transfer Learning Framework Iteratively Training on an Assistant Task for a Target Task |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.17640 |