Towards Understanding Layer Contributions in Tabular In-Context Learning Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917092574363648 |
|---|---|
| author | Balef, Amir Rezaei Koshil, Mykhailo Eggensperger, Katharina |
| author_facet | Balef, Amir Rezaei Koshil, Mykhailo Eggensperger, Katharina |
| contents | Despite the architectural similarities between tabular in-context learning (ICL) models and large language models (LLMs), little is known about how individual layers contribute to tabular prediction. In this paper, we investigate how the latent spaces evolve across layers in tabular ICL models, identify potential redundant layers, and compare these dynamics with those observed in LLMs. We analyze TabPFN and TabICL through the "layers as painters" perspective, finding that only subsets of layers share a common representational language, suggesting structural redundancy and offering opportunities for model compression and improved interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_15432 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Understanding Layer Contributions in Tabular In-Context Learning Models Balef, Amir Rezaei Koshil, Mykhailo Eggensperger, Katharina Machine Learning Artificial Intelligence Despite the architectural similarities between tabular in-context learning (ICL) models and large language models (LLMs), little is known about how individual layers contribute to tabular prediction. In this paper, we investigate how the latent spaces evolve across layers in tabular ICL models, identify potential redundant layers, and compare these dynamics with those observed in LLMs. We analyze TabPFN and TabICL through the "layers as painters" perspective, finding that only subsets of layers share a common representational language, suggesting structural redundancy and offering opportunities for model compression and improved interpretability. |
| title | Towards Understanding Layer Contributions in Tabular In-Context Learning Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.15432 |