Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training

Documenti analoghi

• Scaling Law with Learning Rate Annealing
  di: Tissue, Howe, et al.
  Pubblicazione: (2024)
• Learning Dynamics in Continual Pre-Training for Large Language Models
  di: Wang, Xingjin, et al.
  Pubblicazione: (2025)
• Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
  di: Ye, Jiasheng, et al.
  Pubblicazione: (2024)
• MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining
  di: Wen, Bingbing, et al.
  Pubblicazione: (2026)
• R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training
  di: Ge, Albert, et al.
  Pubblicazione: (2025)