Image de couverture de livre

LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lin, Gang, Li, Dongfang, Chen, Zhuoen, Shi, Yukun, Chen, Xuhui, Hu, Baotian, Zhang, Min
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LycheeCluster: Efficient Long-Context Inference with Structure-Aware Chunking and Hierarchical KV Indexing
par: Li, Dongfang, et autres
Publié: (2026)

Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learning
par: Chen, Zhuoen, et autres
Publié: (2026)

Stabilizing Long-term Multi-turn Reinforcement Learning with Gated Rewards
par: Sun, Zetian, et autres
Publié: (2025)

Is On-Policy Data always the Best Choice for Direct Preference Optimization-based LM Alignment?
par: Sun, Zetian, et autres
Publié: (2025)

Improving Value-based Process Verifier via Low-Cost Variance Reduction
par: Sun, Zetian, et autres
Publié: (2025)

In-Context Learning State Vector with Inner and Momentum Optimization
par: Li, Dongfang, et autres
Publié: (2024)

Temporal Knowledge Question Answering via Abstract Reasoning Induction
par: Chen, Ziyang, et autres
Publié: (2023)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
par: Zhu, Qianchao, et autres
Publié: (2024)

TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
par: Yang, Lijie, et autres
Publié: (2024)

Improving Value-based Process Verifier via Structural Prior Injection
par: Sun, Zetian, et autres
Publié: (2025)

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference
par: Li, Zeping, et autres
Publié: (2024)

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
par: Cai, Tianle, et autres
Publié: (2024)

CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models
par: Li, Dongfang, et autres
Publié: (2024)

Accelerating Transformer Inference for Translation via Parallel Decoding
par: Santilli, Andrea, et autres
Publié: (2023)

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
par: Wen, Zhuofan, et autres
Publié: (2024)

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
par: Timor, Nadav, et autres
Publié: (2025)

Improving Attributed Text Generation of Large Language Models via Preference Learning
par: Li, Dongfang, et autres
Publié: (2024)

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
par: Yao, Jinwei, et autres
Publié: (2024)

Speculative Decoding for Multi-Sample Inference
par: Li, Yiwei, et autres
Publié: (2025)

Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
par: Zhang, Yong, et autres
Publié: (2025)

AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism
par: Wei, Zhepei, et autres
Publié: (2025)

PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
par: Zhang, Hao, et autres
Publié: (2025)

CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
par: Wang, Kangyu, et autres
Publié: (2025)

AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference
par: He, Zhuomin, et autres
Publié: (2025)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
par: Li, Ruixiao, et autres
Publié: (2025)

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
par: Agrawal, Sudhanshu, et autres
Publié: (2025)

SAM Decoding: Speculative Decoding via Suffix Automaton
par: Hu, Yuxuan, et autres
Publié: (2024)

Accelerate Speculative Decoding with Sparse Computation in Verification
par: Wang, Jikai, et autres
Publié: (2025)

RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding
par: Chen, Guanzheng, et autres
Publié: (2025)

Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
par: Peng, Dan, et autres
Publié: (2025)

Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment
par: Liu, Zhenyu, et autres
Publié: (2025)

Hybrid Verified Decoding: Learning to Allocate Verification in Speculative Decoding
par: Su, Xin, et autres
Publié: (2026)

SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
par: Liu, Liangxin, et autres
Publié: (2024)

Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer
par: Hu, Xinshuo, et autres
Publié: (2024)

SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding
par: Sun, Ryan, et autres
Publié: (2024)

Structured Episodic Event Memory
par: Lu, Zhengxuan, et autres
Publié: (2026)

DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations
par: Gema, Aryo Pradipta, et autres
Publié: (2024)

Empirical Analysis of Decoding Biases in Masked Diffusion Models
par: Huang, Pengcheng, et autres
Publié: (2025)

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
par: Yang, Penghui, et autres
Publié: (2025)

Acceleration Multiple Heads Decoding for LLM via Dynamic Tree Attention
par: Zhang, Zhendong
Publié: (2025)