Image de couverture de livre

Faster Cascades via Speculative Decoding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Narasimhan, Harikrishna, Jitkrittum, Wittawat, Rawat, Ankit Singh, Kim, Seungyeon, Gupta, Neha, Menon, Aditya Krishna, Kumar, Sanjiv
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Language Model Cascades: Token-level uncertainty and beyond
par: Gupta, Neha, et autres
Publié: (2024)

When Does Confidence-Based Cascade Deferral Suffice?
par: Jitkrittum, Wittawat, et autres
Publié: (2023)

Cascade-Aware Training of Language Models
par: Wang, Congchao, et autres
Publié: (2024)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
par: Zhou, Yongchao, et autres
Publié: (2023)

Regression-aware Inference with LLMs
par: Lukasik, Michal, et autres
Publié: (2024)

Think before you speak: Training Language Models With Pause Tokens
par: Goyal, Sachin, et autres
Publié: (2023)

Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation
par: Lukasik, Michal, et autres
Publié: (2025)

Universal Model Routing for Efficient LLM Inference
par: Jitkrittum, Wittawat, et autres
Publié: (2025)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
par: Rawat, Ankit Singh, et autres
Publié: (2024)

Self-Speculative Biased Decoding for Faster Re-Translation
par: Zeng, Linxiao, et autres
Publié: (2025)

Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
par: Shoham, Ofir Ben
Publié: (2026)

GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
par: Hu, Shijing, et autres
Publié: (2025)

Dynamic Depth Decoding: Faster Speculative Decoding for LLMs
par: Brown, Oscar, et autres
Publié: (2024)

SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification
par: Yoon, Kanghoon, et autres
Publié: (2025)

A Statistical Framework for Data-dependent Retrieval-Augmented Models
par: Basu, Soumya, et autres
Publié: (2024)

Efficient Document Ranking with Learnable Late Interactions
par: Ji, Ziwei, et autres
Publié: (2024)

HiSpec: Hierarchical Speculative Decoding for LLMs
par: Kumar, Avinash, et autres
Publié: (2025)

Exploring and Improving Drafts in Blockwise Parallel Decoding
par: Kim, Taehyeon, et autres
Publié: (2024)

Online Speculative Decoding
par: Liu, Xiaoxuan, et autres
Publié: (2023)

On student-teacher deviations in distillation: does it pay to disobey?
par: Nagarajan, Vaishnavh, et autres
Publié: (2023)

Mixture of Attentions For Speculative Decoding
par: Zimmer, Matthieu, et autres
Publié: (2024)

Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond
par: Li, Yingcong, et autres
Publié: (2024)

Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
par: Ryu, Hyun, et autres
Publié: (2024)

Fast Large Language Model Collaborative Decoding via Speculation
par: Fu, Jiale, et autres
Publié: (2025)

Traversal Verification for Speculative Tree Decoding
par: Weng, Yepeng, et autres
Publié: (2025)

$\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
par: Cemri, Mert, et autres
Publié: (2025)

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
par: Agrawal, Sudhanshu, et autres
Publié: (2025)

SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
par: Huang, Kaixuan, et autres
Publié: (2024)

Benchmarking the Energy Savings with Speculative Decoding Strategies
par: Dutta, Rohit, et autres
Publié: (2026)

On Speculative Decoding for Multimodal Large Language Models
par: Gagrani, Mukul, et autres
Publié: (2024)

A Theoretical Perspective for Speculative Decoding Algorithm
par: Yin, Ming, et autres
Publié: (2024)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
par: Huang, Haiduo, et autres
Publié: (2025)

Confidence-Modulated Speculative Decoding for Large Language Models
par: Sen, Jaydip, et autres
Publié: (2025)

Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge
par: Xiao, Bin, et autres
Publié: (2024)

From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
par: Ildiz, M. Emrullah, et autres
Publié: (2024)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
par: Hu, Yuezhou, et autres
Publié: (2025)

Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding
par: Xiao, Bin, et autres
Publié: (2024)

POSS: Position Specialist Generates Better Draft for Speculative Decoding
par: Huang, Langlin, et autres
Publié: (2025)

REST: Retrieval-Based Speculative Decoding
par: He, Zhenyu, et autres
Publié: (2023)

Mamba Drafters for Speculative Decoding
par: Choi, Daewon, et autres
Publié: (2025)