Imagen de Portada

Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Sakai, Mana, Imaizumi, Masaaki
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs
por: Sakai, Mana, et al.
Publicado: (2025)

Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size
por: Takeshita, Naoki, et al.
Publicado: (2025)

Automatic Domain Adaptation by Transformers in In-Context Learning
por: Hataya, Ryuichiro, et al.
Publicado: (2024)

Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning
por: Chen, Baiyuan, et al.
Publicado: (2025)

Neuron Block Dynamics for XOR Classification with Zero-Margin
por: Braun, Guillaume, et al.
Publicado: (2026)

High-dimensional Contextual Bandit Problem without Sparsity
por: Komiyama, Junpei, et al.
Publicado: (2023)

Zero Generalization Error Theorem for Random Interpolators via Algebraic Geometry
por: Yoshida, Naoki, et al.
Publicado: (2025)

Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality
por: Tamano, Shu, et al.
Publicado: (2026)

Benign Overfitting in Time Series Linear Models with Over-Parameterization
por: Nakakita, Shogo, et al.
Publicado: (2022)

Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
por: Imaizumi, Masaaki, et al.
Publicado: (2026)

Finite-Sample Inference for Sparsely Permuted Linear Regression
por: Ota, Hirofumi, et al.
Publicado: (2026)

Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra
por: Wakayama, Tomoya, et al.
Publicado: (2023)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
por: Nishiyama, Sota, et al.
Publicado: (2026)

Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
por: Nishiyama, Sota, et al.
Publicado: (2025)

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
por: Isobe, Noboru, et al.
Publicado: (2026)

Minimax Rates of Estimation for Optimal Transport Map between Infinite-Dimensional Spaces
por: Ponnoprat, Donlapark, et al.
Publicado: (2025)

Effect of Weight Quantization on Learning Models by Typical Case Analysis
por: Kashiwamura, Shuhei, et al.
Publicado: (2024)

Effect of Random Learning Rate: Theoretical Analysis of SGD Dynamics in Non-Convex Optimization via Stationary Distribution
por: Yoshida, Naoki, et al.
Publicado: (2024)

Precise gradient descent training dynamics for finite-width multi-layer neural networks
por: Han, Qiyang, et al.
Publicado: (2025)

High-dimensional Nonparametric Contextual Bandit Problem
por: Iwazaki, Shogo, et al.
Publicado: (2025)

Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion
por: Hayakawa, Satoshi, et al.
Publicado: (2025)

Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent
por: Braun, Guillaume, et al.
Publicado: (2025)

Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval
por: Braun, Guillaume, et al.
Publicado: (2026)

Minimax Optimal Estimation of Transport-Growth Pairs in Unbalanced Optimal Transport
por: Ponnoprat, Donlapark, et al.
Publicado: (2026)

Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent
por: Imai, Shota, et al.
Publicado: (2026)

Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data
por: Braun, Guillaume, et al.
Publicado: (2025)

Federated Learning with Relative Fairness
por: Nakakita, Shogo, et al.
Publicado: (2024)

Distillation of Discrete Diffusion through Dimensional Correlations
por: Hayakawa, Satoshi, et al.
Publicado: (2024)

Understanding Generalization in Transformers: Error Bounds and Training Dynamics Under Benign and Harmful Overfitting
por: Zhang, Yingying, et al.
Publicado: (2025)

CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency
por: Ota, Hirofumi, et al.
Publicado: (2026)

Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates
por: Imaizumi, Kento, et al.
Publicado: (2024)

SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer
por: Takida, Yuhta, et al.
Publicado: (2023)

Quantitative Bounds for Length Generalization in Transformers
por: Izzo, Zachary, et al.
Publicado: (2025)

Provable Generalization Bounds for Deep Neural Networks with Momentum-Adaptive Gradient Dropout
por: Safder, Adeel
Publicado: (2025)

Sharper Generalization Bounds for Transformer
por: Li, Yawen, et al.
Publicado: (2026)

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair
por: Sakai, Yusuke, et al.
Publicado: (2024)

Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective
por: Takada, Masaaki, et al.
Publicado: (2023)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
por: Imaizumi, Kento, et al.
Publicado: (2025)

Generalization Bounds for Transformer Channel Decoders
por: Zhang, Qinshan, et al.
Publicado: (2026)

On Rademacher Complexity-based Generalization Bounds for Deep Learning
por: Truong, Lan V.
Publicado: (2022)