Towards Low-bit Communication for Tensor Parallel LLM Inference

Documenti analoghi

• Communication Compression for Tensor Parallel LLM Inference
  di: Hansen-Palmus, Jan, et al.
  Pubblicazione: (2024)
• SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
  di: Kim, Han-Byul, et al.
  Pubblicazione: (2025)
• MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
  di: Zibakhsh, Soheil, et al.
  Pubblicazione: (2025)
• LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
  di: Fu, Qichen, et al.
  Pubblicazione: (2024)
• ICQuant: Index Coding enables Low-bit LLM Quantization
  di: Li, Xinlin, et al.
  Pubblicazione: (2025)