Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-training

Documenti analoghi

• Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
  di: Zhuang, Xinlin, et al.
  Pubblicazione: (2025)
• Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration
  di: Bai, Tianyi, et al.
  Pubblicazione: (2024)
• RegMix: Data Mixture as Regression for Language Model Pre-training
  di: Liu, Qian, et al.
  Pubblicazione: (2024)
• LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
  di: Zhu, Tong, et al.
  Pubblicazione: (2024)
• Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
  di: Fan, Zhiyuan, et al.
  Pubblicazione: (2023)