HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading

Documenti analoghi

• MOM: Memory-Efficient Offloaded Mini-Sequence Inference for Long Context Language Models
  di: Zhang, Junyang, et al.
  Pubblicazione: (2025)
• Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
  di: Luo, Cheng, et al.
  Pubblicazione: (2024)
• GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
  di: Zhao, Jiawei, et al.
  Pubblicazione: (2024)
• InRank: Incremental Low-Rank Learning
  di: Zhao, Jiawei, et al.
  Pubblicazione: (2023)
• EcoSpa: Efficient Transformer Training with Coupled Sparsity
  di: Xiao, Jinqi, et al.
  Pubblicazione: (2025)