Copertina

Make Every Draft Count: Hidden State based Speculative Decoding

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Yuetao, Wang, Xuliang, Zheng, Xinzhou, Li, Ming, Wang, Peng, Xu, Hong
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence Distributed, Parallel, and Cluster Computing Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MineDraft: A Framework for Batch Parallel Speculative Decoding
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
di: Shen, Yuhao, et al.
Pubblicazione: (2025)

FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
di: Li, Yuchen, et al.
Pubblicazione: (2026)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
di: Liu, Xing, et al.
Pubblicazione: (2025)

Make Every Word Count: Adaptive BA with Fewer Words
di: Cohen, Shir, et al.
Pubblicazione: (2022)

SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications
di: Oliaro, Gabriele, et al.
Pubblicazione: (2024)

Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
di: Pan, Rui, et al.
Pubblicazione: (2025)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
di: Li, Zikun, et al.
Pubblicazione: (2025)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
di: Li, Rui, et al.
Pubblicazione: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
di: Song, Jingwei, et al.
Pubblicazione: (2025)

SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
di: Abramovich, Talor, et al.
Pubblicazione: (2026)

Accelerating OpenPangu Inference on NPU via Speculative Decoding
di: Dai, Yuntao, et al.
Pubblicazione: (2026)

Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits
di: Khisti, Ashish, et al.
Pubblicazione: (2024)

AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
di: McDanel, Bradley
Pubblicazione: (2024)

When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMs
di: Wei, Jiankun, et al.
Pubblicazione: (2024)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

SpecMemo: Speculative Decoding is in Your Pocket
di: Yildirim, Selin, et al.
Pubblicazione: (2025)

A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
di: Zhang, Yida, et al.
Pubblicazione: (2026)

Striking the Right Balance between Compute and Copy: Improving LLM Inferencing Under Speculative Decoding
di: Ramachandran, Arun, et al.
Pubblicazione: (2025)

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference
di: Timor, Nadav, et al.
Pubblicazione: (2024)

FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
di: Chen, Wenyan, et al.
Pubblicazione: (2026)

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
di: Feng, Weiqi, et al.
Pubblicazione: (2024)

GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
di: Tran, Phuong, et al.
Pubblicazione: (2025)

SMART: When is it Actually Worth Expanding a Speculative Tree?
di: Wang, Lifu, et al.
Pubblicazione: (2026)

SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection
di: Shukla, Shikhar
Pubblicazione: (2026)

Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits
di: Venkatesha, Yeshwanth, et al.
Pubblicazione: (2025)

Utility-Driven Speculative Decoding for Mixture-of-Experts
di: Saxena, Anish, et al.
Pubblicazione: (2025)

Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
di: Wei, Jinhui, et al.
Pubblicazione: (2025)

PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
di: Yi, Jinjun, et al.
Pubblicazione: (2025)

ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
di: Chen, Qiaoling, et al.
Pubblicazione: (2025)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
di: Xu, Jiaming, et al.
Pubblicazione: (2025)

PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding
di: Han, Yunhe, et al.
Pubblicazione: (2026)

ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
di: Hu, Xinyi, et al.
Pubblicazione: (2026)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
di: Xie, Jincheng, et al.
Pubblicazione: (2026)

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
di: Zheng, Ce, et al.
Pubblicazione: (2026)

Fast LLM Post-training via Decoupled and Fastest-of-N Speculation
di: Cheng, Rongxin, et al.
Pubblicazione: (2025)

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
di: Li, Weiqing, et al.
Pubblicazione: (2025)

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
di: Ma, Qianli, et al.
Pubblicazione: (2025)