The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining

Documenti analoghi

• Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
  di: Bethune, Louis, et al.
  Pubblicazione: (2025)
• The Geometries of Truth Are Orthogonal Across Tasks
  di: Azizian, Waiss, et al.
  Pubblicazione: (2025)
• Nectar: Neural Estimation of Cached-Token Attention via Regression
  di: Monteiro, João, et al.
  Pubblicazione: (2026)
• Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
  di: Ablin, Pierre, et al.
  Pubblicazione: (2025)
• Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
  di: Grangier, David, et al.
  Pubblicazione: (2024)