Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention

Documenti analoghi

• Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers
  di: Súkeník, Peter, et al.
  Pubblicazione: (2025)
• Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?
  di: Súkeník, Peter, et al.
  Pubblicazione: (2024)
• Average gradient outer product as a mechanism for deep neural collapse
  di: Beaglehole, Daniel, et al.
  Pubblicazione: (2024)
• Intriguing Properties of Input-dependent Randomized Smoothing
  di: Súkeník, Peter, et al.
  Pubblicazione: (2021)
• Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
  di: Jacot, Arthur, et al.
  Pubblicazione: (2024)