Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability

Documenti analoghi

• Goldfish: Monolingual Language Models for 350 Languages
  di: Chang, Tyler A., et al.
  Pubblicazione: (2024)
• Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models
  di: Chang, Tyler A., et al.
  Pubblicazione: (2025)
• A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
  di: Arnett, Catherine, et al.
  Pubblicazione: (2024)
• On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
  di: Arnett, Catherine, et al.
  Pubblicazione: (2025)
• Exploring Forgetting in Large Language Model Pre-Training
  di: Liao, Chonghua, et al.
  Pubblicazione: (2024)