Task Addition in Multi-Task Learning by Geometrical Alignment
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909325338869760 |
|---|---|
| author | Yim, Soorin Jeong, Dae-Woong Ko, Sung Moon Lee, Sumin Kim, Hyunseung Lee, Chanhui Han, Sehui |
| author_facet | Yim, Soorin Jeong, Dae-Woong Ko, Sung Moon Lee, Sumin Kim, Hyunseung Lee, Chanhui Han, Sehui |
| contents | Training deep learning models on limited data while maintaining generalization is one of the fundamental challenges in molecular property prediction. One effective solution is transferring knowledge extracted from abundant datasets to those with scarce data. Recently, a novel algorithm called Geometrically Aligned Transfer Encoder (GATE) has been introduced, which uses soft parameter sharing by aligning the geometrical shapes of task-specific latent spaces. However, GATE faces limitations in scaling to multiple tasks due to computational costs. In this study, we propose a task addition approach for GATE to improve performance on target tasks with limited data while minimizing computational complexity. It is achieved through supervised multi-task pre-training on a large dataset, followed by the addition and training of task-specific modules for each target task. Our experiments demonstrate the superior performance of the task addition strategy for GATE over conventional multi-task methods, with comparable computational costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16645 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Task Addition in Multi-Task Learning by Geometrical Alignment Yim, Soorin Jeong, Dae-Woong Ko, Sung Moon Lee, Sumin Kim, Hyunseung Lee, Chanhui Han, Sehui Machine Learning Artificial Intelligence Training deep learning models on limited data while maintaining generalization is one of the fundamental challenges in molecular property prediction. One effective solution is transferring knowledge extracted from abundant datasets to those with scarce data. Recently, a novel algorithm called Geometrically Aligned Transfer Encoder (GATE) has been introduced, which uses soft parameter sharing by aligning the geometrical shapes of task-specific latent spaces. However, GATE faces limitations in scaling to multiple tasks due to computational costs. In this study, we propose a task addition approach for GATE to improve performance on target tasks with limited data while minimizing computational complexity. It is achieved through supervised multi-task pre-training on a large dataset, followed by the addition and training of task-specific modules for each target task. Our experiments demonstrate the superior performance of the task addition strategy for GATE over conventional multi-task methods, with comparable computational costs. |
| title | Task Addition in Multi-Task Learning by Geometrical Alignment |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.16645 |