Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI

Documenti analoghi

• Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
  di: Hidayetoglu, Mert, et al.
  Pubblicazione: (2025)
• Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
  di: Wei, Jinhui, et al.
  Pubblicazione: (2025)
• HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
  di: Lin, Haoran, et al.
  Pubblicazione: (2025)
• SiDP: Memory-Efficient Data Parallelism for Offline LLM Inference
  di: Zhao, Alan, et al.
  Pubblicazione: (2026)
• SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
  di: He, Yongchao, et al.
  Pubblicazione: (2025)