Does your data spark joy? Performance gains from domain upsampling at the end of training

Documenti analoghi

• Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models
  di: Ankner, Zachary, et al.
  Pubblicazione: (2024)
• LoRA Learns Less and Forgets Less
  di: Biderman, Dan, et al.
  Pubblicazione: (2024)
• Soup to go: mitigating forgetting during continual learning with model averaging
  di: Kleiman, Anat, et al.
  Pubblicazione: (2025)
• Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
  di: Sardana, Nikhil, et al.
  Pubblicazione: (2023)
• Scaling Laws for Precision
  di: Kumar, Tanishq, et al.
  Pubblicazione: (2024)