APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs

Documenti analoghi

• APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
  di: Huang, Yuxiang, et al.
  Pubblicazione: (2026)
• Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
  di: Huang, Yuxiang, et al.
  Pubblicazione: (2024)
• InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
  di: Xiao, Chaojun, et al.
  Pubblicazione: (2024)
• BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
  di: Song, Chenyang, et al.
  Pubblicazione: (2025)
• BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
  di: Sun, Ao, et al.
  Pubblicazione: (2024)