Copertina

Speculative Streaming: Fast LLM Inference without Auxiliary Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Bhendawade, Nikhil, Belousova, Irina, Fu, Qichen, Mason, Henry, Rastegari, Mohammad, Najibi, Mahyar
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference
di: Bhendawade, Nikhil, et al.
Pubblicazione: (2025)

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
di: Merth, Thomas, et al.
Pubblicazione: (2024)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
di: Fu, Qichen, et al.
Pubblicazione: (2024)

FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2025)

LEAP: Learnable End-to-End Adaptive Pruning of Large Language Models
di: Mozaffari, Mohammad, et al.
Pubblicazione: (2026)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2026)

DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2026)

OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
di: Mehta, Sachin, et al.
Pubblicazione: (2024)

Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
di: Bhendawade, Nikhil, et al.
Pubblicazione: (2025)

CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
di: Mehta, Sachin, et al.
Pubblicazione: (2024)

SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
di: Shafipour, Rasoul, et al.
Pubblicazione: (2024)

LLM in a flash: Efficient Large Language Model Inference with Limited Memory
di: Alizadeh, Keivan, et al.
Pubblicazione: (2023)

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
di: Wen, Zhuofan, et al.
Pubblicazione: (2024)

Fast Large Language Model Collaborative Decoding via Speculation
di: Fu, Jiale, et al.
Pubblicazione: (2025)

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers
di: Wen, Yuxin, et al.
Pubblicazione: (2024)

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
di: Timor, Nadav, et al.
Pubblicazione: (2025)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
di: Li, Ruixiao, et al.
Pubblicazione: (2025)

PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
di: Wang, Xuliang, et al.
Pubblicazione: (2026)

QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
di: Tiwari, Rishabh, et al.
Pubblicazione: (2025)

Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding
di: Xiao, Bin, et al.
Pubblicazione: (2024)

Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
di: Ryu, Hyun, et al.
Pubblicazione: (2024)

LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
di: Elhoushi, Mostafa, et al.
Pubblicazione: (2024)

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference
di: Timor, Nadav, et al.
Pubblicazione: (2024)

FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference
di: Bajpai, Divya Jyoti, et al.
Pubblicazione: (2025)

xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
di: Beck, Maximilian, et al.
Pubblicazione: (2025)

Optimal Singular Damage: Efficient LLM Inference in Low Storage Regimes
di: Alipour, Mohammadsajad, et al.
Pubblicazione: (2025)

ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity
di: Bae, Henry, et al.
Pubblicazione: (2023)

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
di: Agrawal, Sudhanshu, et al.
Pubblicazione: (2025)

On Speculative Decoding for Multimodal Large Language Models
di: Gagrani, Mukul, et al.
Pubblicazione: (2024)

Block Transformer: Global-to-Local Language Modeling for Fast Inference
di: Ho, Namgyu, et al.
Pubblicazione: (2024)

TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling
di: Qiu, Jiahao, et al.
Pubblicazione: (2024)

Online Speculative Decoding
di: Liu, Xiaoxuan, et al.
Pubblicazione: (2023)

Confidence-Modulated Speculative Decoding for Large Language Models
di: Sen, Jaydip, et al.
Pubblicazione: (2025)

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
di: Zhu, Yuxuan, et al.
Pubblicazione: (2025)

Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
di: Wang, Ziyan, et al.
Pubblicazione: (2025)

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation
di: Qiao, Aurick, et al.
Pubblicazione: (2024)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
di: Cho, Minsik, et al.
Pubblicazione: (2024)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
di: Li, Shuhuai, et al.
Pubblicazione: (2026)

Mixture of Attentions For Speculative Decoding
di: Zimmer, Matthieu, et al.
Pubblicazione: (2024)

PQCache: Product Quantization-based KVCache for Long Context LLM Inference
di: Zhang, Hailin, et al.
Pubblicazione: (2024)