Cover Image

In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamics

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jin, Yanhao, Balasubramanian, Krishnakumar, Lai, Lifeng
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
by: Liang, Haodong, et al.
Published: (2025)

Transformers Handle Endogeneity in In-Context Linear Regression
by: Liang, Haodong, et al.
Published: (2024)

Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation
by: Jin, Yanhao, et al.
Published: (2024)

Large-Step Training Dynamics of a Two-Factor Linear Transformer Model
by: Balasubramanian, Krishnakumar
Published: (2026)

Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression
by: Agrawalla, Bhavya, et al.
Published: (2023)

Nonsmooth Nonparametric Regression via Fractional Laplacian Eigenmaps
by: Shi, Zhaoyang, et al.
Published: (2024)

Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models
by: Balasubramanian, Krishnakumar
Published: (2026)

A Quantitative Characterization of Forgetting in Post-Training
by: Balasubramanian, Krishnakumar, et al.
Published: (2026)

Statistical Inference for Linear Functionals of Online Least-squares SGD when $t \gtrsim d^{1+δ}$
by: Agrawalla, Bhavya, et al.
Published: (2025)

Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains
by: Guan, Yunrui, et al.
Published: (2025)

Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data
by: Chen, Xuxing, et al.
Published: (2024)

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
by: Balasubramanian, Krishnakumar, et al.
Published: (2025)

Dense associative memory for Gaussian distributions
by: Tankala, Chandan, et al.
Published: (2025)

Riemannian Proximal Sampler for High-accuracy Sampling on Manifolds
by: Guan, Yunrui, et al.
Published: (2025)

Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent
by: Banerjee, Sayan, et al.
Published: (2024)

Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models
by: Balasubramanian, Krishnakumar, et al.
Published: (2026)

Total Variation Rates for Riemannian Flow Matching
by: Guan, Yunrui, et al.
Published: (2026)

Minimax Optimal Q Learning with Nearest Neighbors
by: Zhao, Puning, et al.
Published: (2023)

High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance
by: Balasubramanian, Krishnakumar, et al.
Published: (2023)

Automatic Cross-Domain Transfer Learning for Linear Regression
by: Liu, Xinshun, et al.
Published: (2020)

EM Approaches to Nonparametric Estimation for Mixture of Linear Regressions
by: Welbaum, Andrew, et al.
Published: (2025)

Finite-Particle Rates for Regularized Stein Variational Gradient Descent
by: He, Ye, et al.
Published: (2026)

Robust Risk-Sensitive Reinforcement Learning with Conditional Value-at-Risk
by: Ni, Xinyi, et al.
Published: (2024)

Dense Associative Memory with Epanechnikov Energy
by: Hoover, Benjamin, et al.
Published: (2025)

Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
by: Liang, Haodong, et al.
Published: (2026)

Multivariate Gaussian Approximation for Random Forest via Region-based Stabilization
by: Shi, Zhaoyang, et al.
Published: (2024)

Gaussian random field approximation via Stein's method with applications to wide random neural networks
by: Balasubramanian, Krishnakumar, et al.
Published: (2023)

Gaussian and Bootstrap Approximation for Matching-based Average Treatment Effect Estimators
by: Shi, Zhaoyang, et al.
Published: (2024)

Training Dynamics of In-Context Learning in Linear Attention
by: Zhang, Yedi, et al.
Published: (2025)

In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention
by: He, Jianliang, et al.
Published: (2025)

Task Shift: From Classification to Regression in Overparameterized Linear Models
by: LaBonte, Tyler, et al.
Published: (2025)

A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers
by: He, Ye, et al.
Published: (2024)

PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoders
by: Guo, Ruixin, et al.
Published: (2025)

Learn to Change the World: Multi-level Reinforcement Learning with Model-Changing Actions
by: Lu, Ziqing, et al.
Published: (2025)

Online Linear Regression in Dynamic Environments via Discounting
by: Jacobsen, Andrew, et al.
Published: (2024)

Dynamics of Transient Structure in In-Context Linear Regression Transformers
by: Carroll, Liam, et al.
Published: (2025)

Trained Mamba Emulates Online Gradient Descent in In-Context Linear Regression
by: Jiang, Jiarui, et al.
Published: (2025)

Near-Optimal Dynamic Regret for Adversarial Linear Mixture MDPs
by: Li, Long-Fei, et al.
Published: (2024)

Discussion: Effective and Interpretable Outcome Prediction by Training Sparse Mixtures of Linear Experts
by: Folino, Francesco, et al.
Published: (2024)

Provably Invincible Adversarial Attacks on Reinforcement Learning Systems: A Rate-Distortion Information-Theoretic Approach
by: Lu, Ziqing, et al.
Published: (2025)