AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs

Documenti analoghi

• FASTTRACK: Fast and Accurate Fact Tracing for LLMs
  di: Chen, Si, et al.
  Pubblicazione: (2024)
• Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  di: Kang, Feiyang, et al.
  Pubblicazione: (2025)
• Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs
  di: Kang, Feiyang, et al.
  Pubblicazione: (2024)
• AdaDeDup: Adaptive Hybrid Data Pruning for Efficient Large-Scale Object Detection Training
  di: Kang, Feiyang, et al.
  Pubblicazione: (2025)
• SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
  di: Mao, Shizhuo, et al.
  Pubblicazione: (2025)