Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914715109687296 |
|---|---|
| author | Zhang, Zhaoyang Shen, Yantao Shi, Kunyu Cai, Zhaowei Fang, Jun Deng, Siqi Yang, Hao Modolo, Davide Tu, Zhuowen Soatto, Stefano |
| author_facet | Zhang, Zhaoyang Shen, Yantao Shi, Kunyu Cai, Zhaowei Fang, Jun Deng, Siqi Yang, Hao Modolo, Davide Tu, Zhuowen Soatto, Stefano |
| contents | We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks which may interfere with each other, resulting in a single model which we named Musketeer. The integration of knowledge across heterogeneous tasks is enabled by a novel feature called Task Explanation Prompt (TEP). With rich and structured information such as task input/output format, TEP reduces interference among tasks, allowing the model to focus on their shared structure. With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_07019 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts Zhang, Zhaoyang Shen, Yantao Shi, Kunyu Cai, Zhaowei Fang, Jun Deng, Siqi Yang, Hao Modolo, Davide Tu, Zhuowen Soatto, Stefano Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks which may interfere with each other, resulting in a single model which we named Musketeer. The integration of knowledge across heterogeneous tasks is enabled by a novel feature called Task Explanation Prompt (TEP). With rich and structured information such as task input/output format, TEP reduces interference among tasks, allowing the model to focus on their shared structure. With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks. |
| title | Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2305.07019 |