Sparse Layers are Critical to Scaling Looped Language Models

Documenti analoghi

• Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
  di: Sardana, Nikhil, et al.
  Pubblicazione: (2023)
• Scaling Embedding Layers in Language Models
  di: Yu, Da, et al.
  Pubblicazione: (2025)
• Language Models Can Predict Their Own Behavior
  di: Ashok, Dhananjay, et al.
  Pubblicazione: (2025)
• Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
  di: Liu, Xuyuan, et al.
  Pubblicazione: (2025)
• Solve the Loop: Attractor Models for Language and Reasoning
  di: Fein-Ashley, Jacob, et al.
  Pubblicazione: (2026)