Imagen de Portada

Mamba Can Learn Low-Dimensional Targets In-Context via Test-Time Feature Learning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Oh, Junsoo, Huang, Wei, Suzuki, Taiji
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape
por: Kim, Juno, et al.
Publicado: (2024)

Provable Benefit of Cutout and CutMix for Feature Learning
por: Oh, Junsoo, et al.
Publicado: (2024)

Trained Mamba Emulates Online Gradient Descent in In-Context Linear Regression
por: Jiang, Jiarui, et al.
Publicado: (2025)

In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning
por: Wakayama, Tomoya, et al.
Publicado: (2025)

From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning
por: Oh, Junsoo, et al.
Publicado: (2025)

Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input
por: Takakura, Shokichi, et al.
Publicado: (2023)

The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
por: Awano, Ryoya, et al.
Publicado: (2026)

Test time training enhances in-context learning of nonlinear functions
por: Kuwataka, Kento, et al.
Publicado: (2025)

On the Role of Label Noise in the Feature Learning Process
por: Han, Andi, et al.
Publicado: (2025)

How do Transformers perform In-Context Autoregressive Learning?
por: Sander, Michael E., et al.
Publicado: (2024)

Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning
por: Bu, Dake, et al.
Publicado: (2024)

Transformers are Minimax Optimal Nonparametric In-Context Learners
por: Kim, Juno, et al.
Publicado: (2024)

Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency
por: Nishikawa, Naoki, et al.
Publicado: (2025)

Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
por: Park, Jongho, et al.
Publicado: (2024)

Can Mamba Learn In Context with Outliers? A Theoretical Generalization Analysis
por: Li, Hongkang, et al.
Publicado: (2025)

Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples
por: Bu, Dake, et al.
Publicado: (2024)

On the Comparison between Multi-modal and Single-modal Contrastive Learning
por: Huang, Wei, et al.
Publicado: (2024)

How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
por: Higuchi, Rei, et al.
Publicado: (2026)

Provable In-Context Vector Arithmetic via Retrieving Task Concepts
por: Bu, Dake, et al.
Publicado: (2025)

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization
por: Chen, Yihang, et al.
Publicado: (2024)

Is Mamba Capable of In-Context Learning?
por: Grazzi, Riccardo, et al.
Publicado: (2024)

Deep Two-Way Matrix Reordering for Relational Data Analysis
por: Watanabe, Chihiro, et al.
Publicado: (2021)

State Space Models are Provably Comparable to Transformers in Dynamic Token Selection
por: Nishikawa, Naoki, et al.
Publicado: (2024)

Transformers Provably Solve Parity Efficiently with Chain of Thought
por: Kim, Juno, et al.
Publicado: (2024)

Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective
por: Takakura, Shokichi, et al.
Publicado: (2024)

AutoLL: Automatic Linear Layout of Graphs based on Deep Neural Network
por: Watanabe, Chihiro, et al.
Publicado: (2021)

Transformers as Measure-Theoretic Associative Memory: A Statistical Perspective and Minimax Optimality
por: Kawata, Ryotaro, et al.
Publicado: (2026)

Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
por: Kawata, Ryotaro, et al.
Publicado: (2025)

Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations
por: Oko, Kazusato, et al.
Publicado: (2024)

Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models
por: Higuchi, Rei, et al.
Publicado: (2025)

Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression
por: Kim, Juno, et al.
Publicado: (2025)

How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime?
por: Huang, Wei, et al.
Publicado: (2025)

Dimensionality-induced information loss of outliers in deep neural networks
por: Uematsu, Kazuki, et al.
Publicado: (2024)

Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation
por: Kim, Jinuk, et al.
Publicado: (2026)

Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
por: Huang, Wei, et al.
Publicado: (2026)

On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD
por: Zhang, Tongcheng, et al.
Publicado: (2026)

Quantifying the Optimization and Generalization Advantages of Graph Neural Networks Over Multilayer Perceptrons
por: Huang, Wei, et al.
Publicado: (2023)

From Saddle Points Toward Global Minima: A Newton-Type Method on Wasserstein Space
por: Lascu, Razvan-Andrei, et al.
Publicado: (2026)

Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
por: Hübotter, Jonas, et al.
Publicado: (2025)

Can Transformers Break Encryption Schemes via In-Context Learning?
por: Korrapati, Jathin, et al.
Publicado: (2025)