Small Singular Values Matter: A Random Matrix Analysis of Transformer Models

Documenti analoghi

• Boundary between noise and information applied to filtering neural network weight matrices
  di: Staats, Max, et al.
  Pubblicazione: (2022)
• Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning
  di: Staats, Max, et al.
  Pubblicazione: (2023)
• Anti-Correlated Noise in Epoch-Based Stochastic Gradient Descent: Implications for Weight Variances in Flat Directions
  di: Kühn, Marcel, et al.
  Pubblicazione: (2023)
• A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification
  di: Kühn, Marcel, et al.
  Pubblicazione: (2026)
• Unified Description of Learning Dynamics in the Soft Committee Machine from Finite to Ultra-Wide Regimes
  di: Afanah, Assem, et al.
  Pubblicazione: (2025)