Cover Image

Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Zhaiming, Hsu, Alexander, Lai, Rongjie, Liao, Wenjing
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
by: Shen, Zhaiming, et al.
Published: (2025)

Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
by: Hsu, Alexander, et al.
Published: (2026)

Transformers Handle Endogeneity in In-Context Linear Regression
by: Liang, Haodong, et al.
Published: (2024)

Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
by: Chen, Siyu, et al.
Published: (2024)

Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
by: Farghly, Tyler, et al.
Published: (2025)

Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
by: Foster, Dylan J., et al.
Published: (2024)

Conformal Prediction for Privacy-Preserving Machine Learning
by: Balinsky, Alexander David, et al.
Published: (2025)

Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits
by: Chen, Fan, et al.
Published: (2025)

A Fine-Grained Understanding of Uniform Convergence for Halfspaces
by: Kontorovich, Aryeh, et al.
Published: (2026)

Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy
by: Chen, Fan, et al.
Published: (2025)

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators
by: Nguyen, Mike, et al.
Published: (2026)

Scaling Limits of Long-Context Transformers
by: Bruno, Giuseppe, et al.
Published: (2026)

Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
by: Li, Gen, et al.
Published: (2024)

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
by: Lin, Licong, et al.
Published: (2023)

How Particle-System Random Batch Methods Enhance Graph Transformer: Memory Efficiency and Parallel Computing Strategy
by: Liu, Hanwen, et al.
Published: (2025)

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
by: Rajendran, Goutham, et al.
Published: (2024)

Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
by: Liang, Haodong, et al.
Published: (2025)

Compression, Generalization and Learning
by: Campi, Marco C., et al.
Published: (2023)

Online Learning with Unknown Constraints
by: Sridharan, Karthik, et al.
Published: (2024)

Adaptive Sample Aggregation In Transfer Learning
by: Hanneke, Steve, et al.
Published: (2024)

Learning with Differentially Private (Sliced) Wasserstein Gradients
by: Rodríguez-Vítores, David, et al.
Published: (2025)

Provable Reward-Agnostic Preference-Based Reinforcement Learning
by: Zhan, Wenhao, et al.
Published: (2023)

Kernel Two-Sample Tests for Manifold Data
by: Cheng, Xiuyuan, et al.
Published: (2021)

Learning Hierarchical Polynomials of Multiple Nonlinear Features with Three-Layer Networks
by: Fu, Hengyu, et al.
Published: (2024)

Chemical Reaction Networks Learn Better than Spiking Neural Networks
by: Jaffard, Sophie, et al.
Published: (2026)

Beyond identifiability: Learning causal representations with few environments and finite samples
by: Lee, Inbeom, et al.
Published: (2026)

A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)
by: Hazard, Christopher J., et al.
Published: (2025)

A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data
by: Chakraborty, Saptarshi, et al.
Published: (2024)

Labels or Preferences? Budget-Constrained Learning with Human Judgments over AI-Generated Outputs
by: Dong, Zihan, et al.
Published: (2026)

Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions
by: Javanmard, Adel, et al.
Published: (2024)

Towards a Sharp Analysis of Offline Policy Learning for $f$-Divergence-Regularized Contextual Bandits
by: Zhao, Qingyue, et al.
Published: (2025)

Foundations of Structural Causal Models with Latent Selection
by: Chen, Leihao, et al.
Published: (2024)

Unified Algorithms for RL with Decision-Estimation Coefficients: PAC, Reward-Free, Preference-Based Learning, and Beyond
by: Chen, Fan, et al.
Published: (2022)

Near-Optimal Learning and Planning in Separated Latent MDPs
by: Chen, Fan, et al.
Published: (2024)

Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theories
by: Zhang, Haobo, et al.
Published: (2024)

Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inference
by: Golowich, Noah, et al.
Published: (2026)

Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
by: Zhang, Bohan, et al.
Published: (2025)

Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity
by: Levy, Jordan, et al.
Published: (2026)

Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
by: Hu, Xinyang, et al.
Published: (2024)

Le Cam Distortion: A Decision-Theoretic Framework for Robust Transfer Learning
by: Akdemir, Deniz
Published: (2025)