Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking

Documenti analoghi

• Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
  di: Glentis, Athanasios, et al.
  Pubblicazione: (2025)
• Do LLMs Recognize Your Latent Preferences? A Benchmark for Latent Information Discovery in Personalized Interaction
  di: Tsaknakis, Ioannis, et al.
  Pubblicazione: (2025)
• Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
  di: Li, Jiaxiang, et al.
  Pubblicazione: (2024)
• Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates
  di: Glentis, Athanasios, et al.
  Pubblicazione: (2026)
• A Doubly Stochastically Perturbed Algorithm for Linearly Constrained Bilevel Optimization
  di: Khanduri, Prashant, et al.
  Pubblicazione: (2025)