Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915340890406912 |
|---|---|
| author | Ko, Sung Moon Lee, Sumin Jeong, Dae-Woong Lim, Woohyung Han, Sehui |
| author_facet | Ko, Sung Moon Lee, Sumin Jeong, Dae-Woong Lim, Woohyung Han, Sehui |
| contents | Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_06369 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks Ko, Sung Moon Lee, Sumin Jeong, Dae-Woong Lim, Woohyung Han, Sehui Artificial Intelligence Machine Learning Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets. |
| title | Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2310.06369 |