The Role of Sparsity for Length Generalization in Transformers

Documenti analoghi

• Repeat After Me: Transformers are Better than State Space Models at Copying
  di: Jelassi, Samy, et al.
  Pubblicazione: (2024)
• Universal Length Generalization with Turing Programs
  di: Hou, Kaiying, et al.
  Pubblicazione: (2024)
• Loss-to-Loss Prediction: Scaling Laws for All Datasets
  di: Brandfonbrener, David, et al.
  Pubblicazione: (2024)
• LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
  di: Prabhakar, Akshara, et al.
  Pubblicazione: (2024)
• Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones
  di: Mirtaheri, Parsa, et al.
  Pubblicazione: (2025)