Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation

Documenti analoghi

• Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
  di: Giorlandino, Alessio, et al.
  Pubblicazione: (2026)
• Memorisation, convergence and generalisation in generative models
  di: Maillard, Antoine, et al.
  Pubblicazione: (2026)
• A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
  di: Ricci, Fabiola, et al.
  Pubblicazione: (2026)
• Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions
  di: Ricci, Fabiola, et al.
  Pubblicazione: (2025)
• Mapping of attention mechanisms to a generalized Potts model
  di: Rende, Riccardo, et al.
  Pubblicazione: (2023)