Copertina

Provable In-Context Learning of Nonlinear Regression with Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Hongbo, Duan, Lingjie, Liang, Yingbin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

To Theoretically Understand Transformer-Based In-Context Learning for Optimizing CSMA
di: Hao, Shugang, et al.
Pubblicazione: (2025)

Theory on Mixture-of-Experts in Continual Learning
di: Li, Hongbo, et al.
Pubblicazione: (2024)

Theory of Mixture-of-Experts for Mobile Edge Computing
di: Li, Hongbo, et al.
Pubblicazione: (2024)

Agentic Transformers Provably Learn to Search via Reinforcement Learning
di: Yang, Tong, et al.
Pubblicazione: (2026)

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations
di: Huang, Ruiquan, et al.
Pubblicazione: (2023)

Mixture-of-Transformers Learn Faster: A Theoretical Study on Classification Problems
di: Li, Hongbo, et al.
Pubblicazione: (2025)

Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information
di: Cheng, Yuan, et al.
Pubblicazione: (2025)

Multi-head Transformers Provably Learn Symbolic Multi-step Reasoning via Gradient Descent
di: Yang, Tong, et al.
Pubblicazione: (2025)

Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
di: Huang, Ruiquan, et al.
Pubblicazione: (2026)

Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
di: Liang, Haodong, et al.
Pubblicazione: (2026)

In-Context Learning with Representations: Contextual Generalization of Trained Transformers
di: Yang, Tong, et al.
Pubblicazione: (2024)

Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
di: Hsu, Alexander, et al.
Pubblicazione: (2026)

Unlabeled Data Can Provably Enhance In-Context Learning of Transformers
di: Liu, Renpu, et al.
Pubblicazione: (2026)

Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models
di: Liang, Yuchen, et al.
Pubblicazione: (2025)

Non-asymptotic Convergence of Training Transformers for Next-token Prediction
di: Huang, Ruiquan, et al.
Pubblicazione: (2024)

How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias
di: Huang, Ruiquan, et al.
Pubblicazione: (2025)

A Theoretical Analysis of Self-Supervised Learning for Vision Transformers
di: Huang, Yu, et al.
Pubblicazione: (2024)

Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning
di: Bu, Dake, et al.
Pubblicazione: (2024)

How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
di: Li, Hongkang, et al.
Pubblicazione: (2024)

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
di: Li, Zihao, et al.
Pubblicazione: (2024)

Robust Offline Reinforcement Learning for Non-Markovian Decision Processes
di: Huang, Ruiquan, et al.
Pubblicazione: (2024)

Towards Provable Emergence of In-Context Reinforcement Learning
di: Wang, Jiuqi, et al.
Pubblicazione: (2025)

Algorithm Design for Continual Learning in IoT Networks
di: Hao, Shugang, et al.
Pubblicazione: (2024)

When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
di: Li, Hongkang, et al.
Pubblicazione: (2025)

Provable Low-Frequency Bias of In-Context Learning of Representations
di: Yang, Yongyi, et al.
Pubblicazione: (2025)

Transformers Don't In-Context Learn Least Squares Regression
di: Hill, Joshua, et al.
Pubblicazione: (2025)

Transformers Handle Endogeneity in In-Context Linear Regression
di: Liang, Haodong, et al.
Pubblicazione: (2024)

Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
di: Pan, Pei-Chi, et al.
Pubblicazione: (2026)

From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models
di: Liang, Yuchen, et al.
Pubblicazione: (2026)

Transformers Provably Learn to Internalize Chain-of-Thought
di: Huang, Yixiao, et al.
Pubblicazione: (2026)

To Grok Grokking: Provable Grokking in Ridge Regression
di: Xu, Mingyue, et al.
Pubblicazione: (2026)

Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning
di: Li, Yanjie, et al.
Pubblicazione: (2024)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
di: Yang, Hongru, et al.
Pubblicazione: (2024)

Algorithm Design for Online Meta-Learning with Task Boundary Detection
di: Sow, Daouda, et al.
Pubblicazione: (2023)

Transformers Learn Robust In-Context Regression under Distributional Uncertainty
di: Cao, Hoang T. H., et al.
Pubblicazione: (2026)

Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks
di: Nichani, Eshaan, et al.
Pubblicazione: (2023)

Transformers Provably Learn Sparse XOR with Polylogarithmic Parameters
di: Han, Yaomengxi, et al.
Pubblicazione: (2025)

Truthful mechanisms for linear bandit games with private contexts
di: Hu, Yiting, et al.
Pubblicazione: (2025)

Dynamics of Transient Structure in In-Context Linear Regression Transformers
di: Carroll, Liam, et al.
Pubblicazione: (2025)

In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning
di: Wakayama, Tomoya, et al.
Pubblicazione: (2025)