Copertina

Cascade Speculative Drafting for Even Faster LLM Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Ziyi, Yang, Xiaocong, Lin, Jiacheng, Sun, Chenkai, Chang, Kevin Chen-Chuan, Huang, Jie
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
di: Wen, Zhuofan, et al.
Pubblicazione: (2024)

$\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
di: Cemri, Mert, et al.
Pubblicazione: (2025)

Faster Cascades via Speculative Decoding
di: Narasimhan, Harikrishna, et al.
Pubblicazione: (2024)

Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
di: Shoham, Ofir Ben
Pubblicazione: (2026)

PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
di: Wang, Xuliang, et al.
Pubblicazione: (2026)

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
di: Li, Yuhui, et al.
Pubblicazione: (2024)

OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decoding
di: Ramakrishnan, Ramchalam Kinattinkara, et al.
Pubblicazione: (2025)

POSS: Position Specialist Generates Better Draft for Speculative Decoding
di: Huang, Langlin, et al.
Pubblicazione: (2025)

TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMs
di: Lee, Minjae, et al.
Pubblicazione: (2026)

Faster LLM Inference via Sequential Monte Carlo
di: Emara, Yahya, et al.
Pubblicazione: (2026)

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference
di: Timor, Nadav, et al.
Pubblicazione: (2024)

Faster MoE LLM Inference for Extremely Large Models
di: Yang, Haoqi, et al.
Pubblicazione: (2025)

SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding
di: Plaksin, Anton, et al.
Pubblicazione: (2026)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
di: Li, Ruixiao, et al.
Pubblicazione: (2025)

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
di: Yang, Penghui, et al.
Pubblicazione: (2025)

Make Every Draft Count: Hidden State based Speculative Decoding
di: Chen, Yuetao, et al.
Pubblicazione: (2026)

ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
di: Georganas, Evangelos, et al.
Pubblicazione: (2025)

Training Domain Draft Models for Speculative Decoding: Best Practices and Insights
di: Hong, Fenglu, et al.
Pubblicazione: (2025)

Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
di: Bachmann, Gregor, et al.
Pubblicazione: (2025)

Self-Speculative Biased Decoding for Faster Re-Translation
di: Zeng, Linxiao, et al.
Pubblicazione: (2025)

Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs
di: Goel, Raghavv, et al.
Pubblicazione: (2024)

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
di: Sun, Hanshi, et al.
Pubblicazione: (2024)

Speculative Streaming: Fast LLM Inference without Auxiliary Models
di: Bhendawade, Nikhil, et al.
Pubblicazione: (2024)

Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
di: Sun, Shuoyang, et al.
Pubblicazione: (2026)

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
di: Zhao, Weilin, et al.
Pubblicazione: (2024)

FlashDecoding++: Faster Large Language Model Inference on GPUs
di: Hong, Ke, et al.
Pubblicazione: (2023)

MineDraft: A Framework for Batch Parallel Speculative Decoding
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits
di: Khisti, Ashish, et al.
Pubblicazione: (2024)

FastEagle: Cascaded Drafting for Accelerating Speculative Decoding
di: Huang, Haiduo, et al.
Pubblicazione: (2025)

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
di: Timor, Nadav, et al.
Pubblicazione: (2025)

TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling
di: Qiu, Jiahao, et al.
Pubblicazione: (2024)

Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding
di: Xiao, Bin, et al.
Pubblicazione: (2024)

PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
di: Butler, Branden, et al.
Pubblicazione: (2024)

Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum
di: Pouransari, Hadi, et al.
Pubblicazione: (2024)

AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability
di: Agrawal, Sudhanshu, et al.
Pubblicazione: (2024)

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
di: Huang, Jie, et al.
Pubblicazione: (2023)

Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
di: Zhou, Xuwen, et al.
Pubblicazione: (2026)

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
di: Sun, Hanshi, et al.
Pubblicazione: (2024)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
di: Liu, Di, et al.
Pubblicazione: (2024)

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
di: Tseng, Albert, et al.
Pubblicazione: (2024)