Imagen de Portada

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Huang, Yuxiang, Li, Mingye, Han, Xu, Xiao, Chaojun, Zhao, Weilin, Ao, Sun, Zhou, Hao, Zhou, Jie, Liu, Zhiyuan, Sun, Maosong
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
por: Huang, Yuxiang, et al.
Publicado: (2026)

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
por: Huang, Yuxiang, et al.
Publicado: (2024)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
por: Xiao, Chaojun, et al.
Publicado: (2024)

BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
por: Song, Chenyang, et al.
Publicado: (2025)

BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
por: Sun, Ao, et al.
Publicado: (2024)

Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
por: Wang, Xing, et al.
Publicado: (2025)

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
por: Xiao, Chaojun, et al.
Publicado: (2023)

BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
por: Sun, Ao, et al.
Publicado: (2025)

SparseAccelerate: Efficient Long-Context Inference for Mid-Range GPUs
por: Vo, James
Publicado: (2024)

Densing Law of LLMs
por: Xiao, Chaojun, et al.
Publicado: (2024)

InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
por: Zhao, Weilin, et al.
Publicado: (2025)

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
por: Zhao, Weilin, et al.
Publicado: (2024)

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
por: Long, Lingkun, et al.
Publicado: (2026)

SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruning
por: Long, Lingkun, et al.
Publicado: (2025)

From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
por: Zhou, Yiqing, et al.
Publicado: (2025)

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
por: Zhao, Weilin, et al.
Publicado: (2025)

Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs
por: Synk, Ryan, et al.
Publicado: (2025)

CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
por: Zhao, Weilin, et al.
Publicado: (2023)

NOSA: Native and Offloadable Sparse Attention
por: Huang, Yuxiang, et al.
Publicado: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
por: Chen, Yingfa, et al.
Publicado: (2025)

ContextPilot: Fast Long-Context Inference via Context Reuse
por: Jiang, Yinsicheng, et al.
Publicado: (2025)

Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning
por: Wang, Xiaorong, et al.
Publicado: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
por: Chen, Yingfa, et al.
Publicado: (2026)

Exploring the Benefit of Activation Sparsity in Pre-training
por: Zhang, Zhengyan, et al.
Publicado: (2024)

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
por: Sun, Ao, et al.
Publicado: (2024)

H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
por: Gao, Cheng, et al.
Publicado: (2025)

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs
por: Gao, Cheng, et al.
Publicado: (2024)

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
por: Zhang, Xinrong, et al.
Publicado: (2024)

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression
por: Wang, Wenshan, et al.
Publicado: (2024)

Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference
por: Fei, Weizhi, et al.
Publicado: (2025)

Characterizing Prompt Compression Methods for Long Context Inference
por: Jha, Siddharth, et al.
Publicado: (2024)

VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inference
por: Liu, Anmin, et al.
Publicado: (2026)

A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
por: Ge, Suyu, et al.
Publicado: (2024)

FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection
por: Huang, Yufei, et al.
Publicado: (2024)

Context Forcing: Consistent Autoregressive Video Generation with Long Context
por: Chen, Shuo, et al.
Publicado: (2026)

Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
por: Bu, Tao, et al.
Publicado: (2025)

DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
por: Song, Chenyang, et al.
Publicado: (2026)

Long Context Compression with Activation Beacon
por: Zhang, Peitian, et al.
Publicado: (2024)

RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
por: Behnam, Payman, et al.
Publicado: (2025)

UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference
por: Zhou, Lang, et al.
Publicado: (2026)