Cover Image

Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kwon, Soo Min, Xu, Alec S., Yaras, Can, Balzano, Laura, Qu, Qing
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations
by: Xu, Alec S., et al.
Published: (2026)

Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation
by: Yaras, Can, et al.
Published: (2024)

Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics
by: Kwon, Soo Min, et al.
Published: (2023)

MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention
by: Yaras, Can, et al.
Published: (2025)

An Overview of Low-Rank Structures in the Training and Adaptation of Large Models
by: Balzano, Laura, et al.
Published: (2025)

Principled Out-of-Distribution Generalization via Simplicity
by: Ge, Jiawei, et al.
Published: (2025)

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions
by: Wu, Jingda, et al.
Published: (2026)

Linearly Separable Features in Shallow Nonlinear Networks: Width Scales Polynomially with Intrinsic Data Dimension
by: Xu, Alec S., et al.
Published: (2025)

Distributionally Robust Instrumental Variables Estimation
by: Qu, Zhaonan, et al.
Published: (2024)

Optimal Ridge Regularization for Out-of-Distribution Prediction
by: Patil, Pratik, et al.
Published: (2024)

Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data
by: Cribeiro-Ramallo, Jose, et al.
Published: (2025)

Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspective
by: Roy, Saptarshi, et al.
Published: (2026)

Invariant Subspace Decomposition
by: Lazzaretto, Margherita, et al.
Published: (2024)

Theoretical Guarantees for the Subspace-Constrained Tyler's Estimator
by: Lerman, Gilad, et al.
Published: (2024)

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
by: Wang, Peng, et al.
Published: (2023)

A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities
by: Dandi, Yatin, et al.
Published: (2024)

Distributional Treatment Effect Estimation across Heterogeneous Sites via Optimal Transport
by: Bateni, Borna, et al.
Published: (2025)

Transformers Meet In-Context Learning: A Universal Approximation Theory
by: Li, Gen, et al.
Published: (2025)

Learning Survival Distributions with the Asymmetric Laplace Distribution
by: Sheng, Deming, et al.
Published: (2025)

The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model
by: Shi, Laixi, et al.
Published: (2023)

Leave-one-out Singular Subspace Perturbation Analysis for Spectral Clustering
by: Zhang, Anderson Y., et al.
Published: (2022)

Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
by: Kim, Yeichan, et al.
Published: (2025)

On the Role of Depth and Looping for In-Context Learning with Task Diversity
by: Gatmiry, Khashayar, et al.
Published: (2024)

A Conditional Distribution Equality Testing Framework using Deep Generative Learning
by: Zheng, Siming, et al.
Published: (2025)

Learning Confidence Ellipsoids and Applications to Robust Subspace Recovery
by: Gao, Chao, et al.
Published: (2025)

Bias-Corrected Joint Spectral Embedding for Multilayer Networks with Invariant Subspace: Entrywise Eigenvector Perturbation and Inference
by: Xie, Fangzheng
Published: (2024)

Optimal Convergence Analysis of DDPM for General Distributions
by: Jiao, Yuchen, et al.
Published: (2025)

Statistical Learning Theory for Distributional Classification
by: Fiedler, Christian
Published: (2026)

Distributionally-Constrained Adversaries in Online Learning
by: Blanchard, Moïse, et al.
Published: (2025)

A Generalized Adaptive Joint Learning Framework for High-Dimensional Time-Varying Models
by: Chen, Baolin, et al.
Published: (2026)

Optimal Estimation of Shared Singular Subspaces across Multiple Noisy Matrices
by: Ma, Zhengchi, et al.
Published: (2024)

Byzantine-Robust Distributed Sparse Learning Revisited
by: Wang, Yuxuan, et al.
Published: (2026)

Provable Efficiency of Guidance in Diffusion Models for General Data Distribution
by: Li, Gen, et al.
Published: (2025)

A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation
by: Ryu, J. Jon, et al.
Published: (2024)

Deep Distributional Learning with Non-crossing Quantile Network
by: Shen, Guohao, et al.
Published: (2025)

Robust Learning of Multi-index Models via Iterative Subspace Approximation
by: Diakonikolas, Ilias, et al.
Published: (2025)

A General Framework for Treatment Effect Estimation in Semi-Supervised and High Dimensional Settings
by: Chakrabortty, Abhishek, et al.
Published: (2022)

Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution
by: Vardanyan, Elen, et al.
Published: (2023)

Sparsified-Learning for High-Dimensional Heavy-Tailed Locally Stationary Time Series, Concentration and Oracle Inequalities
by: Wang, Yingjie, et al.
Published: (2025)

Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
by: Li, Gen, et al.
Published: (2024)