Copertina

Understanding Generalization in Transformers: Error Bounds and Training Dynamics Under Benign and Harmful Overfitting

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Yingying, Wu, Zhenyu, Li, Jian, Liu, Yong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness
di: Hao, Yifan, et al.
Pubblicazione: (2024)

Benign Overfitting in Adversarial Training for Vision Transformers
di: Zhang, Jiaming, et al.
Pubblicazione: (2026)

Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization
di: Jiang, Jiarui, et al.
Pubblicazione: (2024)

Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context
di: Frei, Spencer, et al.
Pubblicazione: (2024)

Provable Weak-to-Strong Generalization via Benign Overfitting
di: Wu, David X., et al.
Pubblicazione: (2024)

When Does $\ell_2$-Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the $\ell_1$ Implicit Bias
di: Su, Ye, et al.
Pubblicazione: (2026)

Benign Overfitting in Out-of-Distribution Generalization of Linear Models
di: Tang, Shange, et al.
Pubblicazione: (2024)

Benign Overfitting with Quantum Kernels
di: Tomasi, Joachim, et al.
Pubblicazione: (2025)

Benign Overfitting in Single-Head Attention
di: Magen, Roey, et al.
Pubblicazione: (2024)

Harmful Overfitting in Sobolev Spaces
di: Karhadkar, Kedar, et al.
Pubblicazione: (2026)

Benign Overfitting in Token Selection of Attention Mechanism
di: Sakamoto, Keitaro, et al.
Pubblicazione: (2024)

Benign Overfitting for Regression with Trained Two-Layer ReLU Networks
di: Park, Junhyung, et al.
Pubblicazione: (2024)

Rethinking Benign Overfitting in Two-Layer Neural Networks
di: Xu, Ruichen, et al.
Pubblicazione: (2025)

Benign Overfitting in Linear Classifiers with a Bias Term
di: Kondo, Yuta
Pubblicazione: (2025)

Beyond Benign Overfitting in Nadaraya-Watson Interpolators
di: Barzilai, Daniel, et al.
Pubblicazione: (2025)

Benign Overfitting and the Geometry of the Ridge Regression Solution in Binary Classification
di: Tsigler, Alexander, et al.
Pubblicazione: (2025)

Universality of Benign Overfitting in Binary Linear Classification
di: Hashimoto, Ichiro, et al.
Pubblicazione: (2025)

Understanding Robust Overfitting from the Feature Generalization Perspective
di: Yu, Chaojian, et al.
Pubblicazione: (2023)

A Classical View on Benign Overfitting: The Role of Sample Size
di: Park, Junhyung, et al.
Pubblicazione: (2025)

Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
di: Kim, Yeichan, et al.
Pubblicazione: (2025)

Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions
di: Lu, Weihao, et al.
Pubblicazione: (2026)

Benign Overfitting in Time Series Linear Models with Over-Parameterization
di: Nakakita, Shogo, et al.
Pubblicazione: (2022)

Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
di: Mallinar, Neil, et al.
Pubblicazione: (2022)

On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics
di: Li, Binghui, et al.
Pubblicazione: (2023)

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
di: Li, Jiping, et al.
Pubblicazione: (2025)

Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data
di: Frei, Spencer, et al.
Pubblicazione: (2022)

From Tempered to Benign Overfitting in ReLU Neural Networks
di: Kornowski, Guy, et al.
Pubblicazione: (2023)

Benign Overfitting under Learning Rate Conditions for $α$ Sub-exponential Input
di: Okudo, Kota, et al.
Pubblicazione: (2024)

On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training
di: Liu, Chen, et al.
Pubblicazione: (2021)

Initialization Matters: On the Benign Overfitting of Two-Layer ReLU CNN with Fully Trainable Layers
di: Shang, Shuning, et al.
Pubblicazione: (2024)

Spectral-Transport Stability and Benign Overfitting in Interpolating Learning
di: Fredriksson-Imanov, Gustav Olaf Yunus Laitinen-Lundström
Pubblicazione: (2026)

Relative Overfitting and Accept-Reject Framework
di: Liu, Yanxin, et al.
Pubblicazione: (2025)

Friend or Foe? Harnessing Controllable Overfitting for Anomaly Detection
di: Qian, Long, et al.
Pubblicazione: (2024)

Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers
di: Sakai, Mana, et al.
Pubblicazione: (2026)

Generalization Bounds for Transformer Channel Decoders
di: Zhang, Qinshan, et al.
Pubblicazione: (2026)

Understanding the Training and Generalization of Pretrained Transformer for Sequential Decision Making
di: Wang, Hanzhao, et al.
Pubblicazione: (2024)

Error Bounds For Gaussian Process Regression Under Bounded Support Noise With Applications To Safety Certification
di: Reed, Robert, et al.
Pubblicazione: (2024)

Testing for Overfitting
di: Schmidt, James
Pubblicazione: (2023)

Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks
di: Hashimoto, Ichiro
Pubblicazione: (2025)

On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
di: Ye, Hao, et al.
Pubblicazione: (2026)