BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batching

Documenti analoghi

• Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
  di: Cong, Peizhuang, et al.
  Pubblicazione: (2025)
• MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Training
  di: Luo, Yang, et al.
  Pubblicazione: (2025)
• Prediction Is All MoE Needs: Expert Load Distribution Goes from Fluctuating to Stabilizing
  di: Cong, Peizhuang, et al.
  Pubblicazione: (2024)
• BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching
  di: Zhao, Yilong, et al.
  Pubblicazione: (2024)
• Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in $\{\pm 1, \pm i\}$
  di: Wang, Feiyu, et al.
  Pubblicazione: (2025)