Mixture of Parrots: Experts improve memorization more than reasoning

Documenti analoghi

• Repeat After Me: Transformers are Better than State Space Models at Copying
  di: Jelassi, Samy, et al.
  Pubblicazione: (2024)
• Loss-to-Loss Prediction: Scaling Laws for All Datasets
  di: Brandfonbrener, David, et al.
  Pubblicazione: (2024)
• Universal Length Generalization with Turing Programs
  di: Hou, Kaiying, et al.
  Pubblicazione: (2024)
• The Role of Sparsity for Length Generalization in Transformers
  di: Golowich, Noah, et al.
  Pubblicazione: (2025)
• LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
  di: Prabhakar, Akshara, et al.
  Pubblicazione: (2024)