Procedural Pretraining: Warming Up Language Models with Abstract Data

Documenti analoghi

• Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning
  di: Shinnick, Zachary, et al.
  Pubblicazione: (2025)
• Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers
  di: Shinnick, Zachary, et al.
  Pubblicazione: (2025)
• Leaner Transformers: More Heads, Less Depth
  di: Saratchandran, Hemanth, et al.
  Pubblicazione: (2025)
• Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
  di: Albert, Paul, et al.
  Pubblicazione: (2025)
• OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?
  di: Jiang, Liangze, et al.
  Pubblicazione: (2024)