Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.14880 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918251737382912 |
|---|---|
| author | Brien, Darrin O' Gajulapalli, Dhikshith Xia, Eric |
| author_facet | Brien, Darrin O' Gajulapalli, Dhikshith Xia, Eric |
| contents | Results in interpretability suggest that large vision and language models learn implicit linear encodings when models are biased by in-context prompting. However, the existence of similar linear representations in more general adaptation regimes has not yet been demonstrated. In this work, we develop the concept of a task matrix, a linear transformation from a base to finetuned embedding state. We demonstrate that for vision and text models and ten different datasets, a base model augmented with a task matrix achieves results surpassing linear probes, sometimes approaching finetuned levels. Our results validate the existence of cross-layer linear encodings between pretrained and finetuned architectures. Moreover, we show that a data-based approximation for such encodings is both efficient and generalizable to multiple domains. We make our implementation publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14880 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Task Matrices: Linear Maps for Cross-Model Finetuning Transfer Brien, Darrin O' Gajulapalli, Dhikshith Xia, Eric Machine Learning Computation and Language Computer Vision and Pattern Recognition Results in interpretability suggest that large vision and language models learn implicit linear encodings when models are biased by in-context prompting. However, the existence of similar linear representations in more general adaptation regimes has not yet been demonstrated. In this work, we develop the concept of a task matrix, a linear transformation from a base to finetuned embedding state. We demonstrate that for vision and text models and ten different datasets, a base model augmented with a task matrix achieves results surpassing linear probes, sometimes approaching finetuned levels. Our results validate the existence of cross-layer linear encodings between pretrained and finetuned architectures. Moreover, we show that a data-based approximation for such encodings is both efficient and generalizable to multiple domains. We make our implementation publicly available. |
| title | Task Matrices: Linear Maps for Cross-Model Finetuning Transfer |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.14880 |