Image de couverture de livre

MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Sadhukhan, Ranajoy, Chen, Jian, Chen, Zhuoming, Tiwari, Vashisth, Lai, Ruihang, Shi, Jinyuan, Yen, Ian En-Hsu, May, Avner, Chen, Tianqi, Chen, Beidi
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
par: Svirschevski, Ruslan, et autres
Publié: (2024)

Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
par: Chen, Zhuoming, et autres
Publié: (2024)

Kinetics: Rethinking Test-Time Scaling Laws
par: Sadhukhan, Ranajoy, et autres
Publié: (2025)

MagicPIG: LSH Sampling for Efficient LLM Generation
par: Chen, Zhuoming, et autres
Publié: (2024)

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
par: Sun, Hanshi, et autres
Publié: (2024)

Speculative Speculative Decoding
par: Kumar, Tanishq, et autres
Publié: (2026)

Memory Mosaics
par: Zhang, Jianyu, et autres
Publié: (2024)

Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
par: Song, Jingwei, et autres
Publié: (2025)

WWW.Serve: Interconnecting Global LLM Services through Decentralization
par: Wang, Huanyu, et autres
Publié: (2026)

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding
par: Yang, Xinyu, et autres
Publié: (2025)

STEM: Scaling Transformers with Embedding Modules
par: Sadhukhan, Ranajoy, et autres
Publié: (2026)

Nearest Neighbor Speculative Decoding for LLM Generation and Attribution
par: Li, Minghan, et autres
Publié: (2024)

GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?
par: Zhou, Yang, et autres
Publié: (2025)

Sirius: Contextual Sparsity with Correction for Efficient LLMs
par: Zhou, Yang, et autres
Publié: (2024)

Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
par: Luo, Cheng, et autres
Publié: (2024)

Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch Reinforcement Learning
par: Chen, Zhuoming, et autres
Publié: (2026)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
par: Li, Ruixiao, et autres
Publié: (2025)

Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation
par: Liu, Jingyu, et autres
Publié: (2025)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
par: Li, Zikun, et autres
Publié: (2025)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
par: Agrawal, Amey, et autres
Publié: (2024)

FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
par: Yan, Ran, et autres
Publié: (2025)

Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
par: Yang, Xinyu, et autres
Publié: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
par: Wang, Zhibin, et autres
Publié: (2025)

Reliability-Latency-Rate Tradeoff in Low-Latency Communications with Finite-Blocklength Coding
par: Li, Lintao, et autres
Publié: (2023)

SSR: Spatial Sequential Hybrid Architecture for Latency Throughput Tradeoff in Transformer Acceleration
par: Zhuang, Jinming, et autres
Publié: (2024)

LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
par: Elhoushi, Mostafa, et autres
Publié: (2024)

Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff
par: Holsman, Maximilian, et autres
Publié: (2025)

SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
par: Huang, Kaixuan, et autres
Publié: (2024)

SPIRe: Boosting LLM Inference Throughput with Speculative Decoding
par: Neelam, Sanjit, et autres
Publié: (2025)

An Interpretable Latency Model for Speculative Decoding in LLM Serving
par: Kong, Linghao, et autres
Publié: (2026)

MonarchRT: Efficient Attention for Real-Time Video Generation
par: Agarwal, Krish, et autres
Publié: (2026)

SpecMemo: Speculative Decoding is in Your Pocket
par: Yildirim, Selin, et autres
Publié: (2025)

Multi-Candidate Speculative Decoding
par: Yang, Sen, et autres
Publié: (2024)

Scaling Laws for Speculative Decoding
par: Yan, Siyuan, et autres
Publié: (2025)

DFlash: Block Diffusion for Flash Speculative Decoding
par: Chen, Jian, et autres
Publié: (2026)

SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
par: Zhang, Ziyi, et autres
Publié: (2025)

Speculative Decoding Scaling Laws (SDSL): Throughput Optimization Made Simple
par: Bozorgkhoo, Amirhossein, et autres
Publié: (2026)

Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
par: Shoham, Ofir Ben
Publié: (2026)

A Theoretical Perspective for Speculative Decoding Algorithm
par: Yin, Ming, et autres
Publié: (2024)

CoDec: Prefix-Shared Decoding Kernel for LLMs
par: Wang, Zhibin, et autres
Publié: (2025)