Copertina

Transformers as Measure-Theoretic Associative Memory: A Statistical Perspective and Minimax Optimality

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kawata, Ryotaro, Suzuki, Taiji
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Transformers are Minimax Optimal Nonparametric In-Context Learners
di: Kim, Juno, et al.
Pubblicazione: (2024)

Direct Distributional Optimization for Provable Alignment of Diffusion Models
di: Kawata, Ryotaro, et al.
Pubblicazione: (2025)

Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
di: Kawata, Ryotaro, et al.
Pubblicazione: (2025)

From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers
di: Kawata, Ryotaro, et al.
Pubblicazione: (2025)

How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
di: Higuchi, Rei, et al.
Pubblicazione: (2026)

Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models
di: Higuchi, Rei, et al.
Pubblicazione: (2025)

Transformers Provably Solve Parity Efficiently with Chain of Thought
di: Kim, Juno, et al.
Pubblicazione: (2024)

Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective
di: Takakura, Shokichi, et al.
Pubblicazione: (2024)

State Space Models are Provably Comparable to Transformers in Dynamic Token Selection
di: Nishikawa, Naoki, et al.
Pubblicazione: (2024)

Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input
di: Takakura, Shokichi, et al.
Pubblicazione: (2023)

Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape
di: Kim, Juno, et al.
Pubblicazione: (2024)

Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems
di: Kim, Juno, et al.
Pubblicazione: (2023)

Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency
di: Nishikawa, Naoki, et al.
Pubblicazione: (2025)

In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning
di: Wakayama, Tomoya, et al.
Pubblicazione: (2025)

When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
di: Higuchi, Rei, et al.
Pubblicazione: (2025)

The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
di: Awano, Ryoya, et al.
Pubblicazione: (2026)

Deep Two-Way Matrix Reordering for Relational Data Analysis
di: Watanabe, Chihiro, et al.
Pubblicazione: (2021)

AutoLL: Automatic Linear Layout of Graphs based on Deep Neural Network
di: Watanabe, Chihiro, et al.
Pubblicazione: (2021)

Test time training enhances in-context learning of nonlinear functions
di: Kuwataka, Kento, et al.
Pubblicazione: (2025)

On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality
di: Hu, Jerry Yao-Chieh, et al.
Pubblicazione: (2024)

Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression
di: Kim, Juno, et al.
Pubblicazione: (2025)

From Saddle Points Toward Global Minima: A Newton-Type Method on Wasserstein Space
di: Lascu, Razvan-Andrei, et al.
Pubblicazione: (2026)

Mamba Can Learn Low-Dimensional Targets In-Context via Test-Time Feature Learning
di: Oh, Junsoo, et al.
Pubblicazione: (2025)

Understanding Transformer from the Perspective of Associative Memory
di: Zhong, Shu, et al.
Pubblicazione: (2025)

How do Transformers perform In-Context Autoregressive Learning?
di: Sander, Michael E., et al.
Pubblicazione: (2024)

Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
di: Naganuma, Hiroki, et al.
Pubblicazione: (2026)

Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
di: Yamamoto, Naoya, et al.
Pubblicazione: (2025)

Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric
di: Uesaka, Toshimitsu, et al.
Pubblicazione: (2024)

On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent
di: Li, Bingrui, et al.
Pubblicazione: (2024)

Minimax Optimal Reinforcement Learning with Quasi-Optimism
di: Lee, Harin, et al.
Pubblicazione: (2025)

Linear Bandits on Ellipsoids: Minimax Optimal Algorithms
di: Zhang, Raymond, et al.
Pubblicazione: (2025)

Minimax Optimal Q Learning with Nearest Neighbors
di: Zhao, Puning, et al.
Pubblicazione: (2023)

Zero-Flow Encoders
di: Wang, Yakun, et al.
Pubblicazione: (2026)

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization
di: Chen, Yihang, et al.
Pubblicazione: (2024)

Towards a Unified Analysis of Neural Networks in Nonparametric Instrumental Variable Regression: Optimization and Generalization
di: Chen, Zonghao, et al.
Pubblicazione: (2025)

Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations
di: Oko, Kazusato, et al.
Pubblicazione: (2024)

Pretrained transformer efficiently learns low-dimensional target functions in-context
di: Oko, Kazusato, et al.
Pubblicazione: (2024)

Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization
di: Jiang, Jiarui, et al.
Pubblicazione: (2024)

Efficient and Minimax Optimal In-context Nonparametric Regression with Transformers
di: Ching, Michelle, et al.
Pubblicazione: (2026)

Minimax Statistical Estimation under Wasserstein Contamination
di: Chao, Patrick, et al.
Pubblicazione: (2023)