Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.03644 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917829796691968 |
|---|---|
| author | Qu, Yincen Ma, Chao Dai, Xiangying Zhou, Hui Wu, Yiting Liu, Hengyue |
| author_facet | Qu, Yincen Ma, Chao Dai, Xiangying Zhou, Hui Wu, Yiting Liu, Hengyue |
| contents | In the industry, numerous tasks are deployed online. Traditional approaches often tackle each task separately by its own network, which leads to excessive costs for developing and scaling models, especially in the context of large language models. Although multi-task methods can save costs through parameter sharing, they often struggle to outperform single-task methods in real-world applications. To tackle these challenges, we present a three-stage multi-task learning framework for large language models. It involves task filtering, followed by fine-tuning on high-resource tasks, and finally fine-tuning on all tasks. We conducted comprehensive experiments in single-task and multi-task settings. Our approach, exemplified on different benchmarks, demonstrates that it is able to achieve performance comparable to the single-task method while reducing up to 90.9\% of its overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03644 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deploying Multi-task Online Server with Large Language Model Qu, Yincen Ma, Chao Dai, Xiangying Zhou, Hui Wu, Yiting Liu, Hengyue Computation and Language Artificial Intelligence In the industry, numerous tasks are deployed online. Traditional approaches often tackle each task separately by its own network, which leads to excessive costs for developing and scaling models, especially in the context of large language models. Although multi-task methods can save costs through parameter sharing, they often struggle to outperform single-task methods in real-world applications. To tackle these challenges, we present a three-stage multi-task learning framework for large language models. It involves task filtering, followed by fine-tuning on high-resource tasks, and finally fine-tuning on all tasks. We conducted comprehensive experiments in single-task and multi-task settings. Our approach, exemplified on different benchmarks, demonstrates that it is able to achieve performance comparable to the single-task method while reducing up to 90.9\% of its overhead. |
| title | Deploying Multi-task Online Server with Large Language Model |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.03644 |