Image de couverture de livre

CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhou, Enyu, Sheng, Kai, Chen, Hao, He, Xin
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding
par: Yin, Haofei, et autres
Publié: (2025)

Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
par: Jeon, Wonseok, et autres
Publié: (2024)

Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices
par: Mesa, Alejandro Ruiz y, et autres
Publié: (2026)

Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference
par: Chen, Hao Mark, et autres
Publié: (2024)

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
par: Wen, Zhuofan, et autres
Publié: (2024)

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
par: Timor, Nadav, et autres
Publié: (2025)

SPIRe: Boosting LLM Inference Throughput with Speculative Decoding
par: Neelam, Sanjit, et autres
Publié: (2025)

RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
par: Geng, Yingsheng, et autres
Publié: (2026)

SNLP: Layer-Parallel Inference via Structured Newton Corrections
par: Han, Ligong, et autres
Publié: (2026)

Collaborative Large Language Model Inference via Resource-Aware Parallel Speculative Decoding
par: Koh, Jungyeon, et autres
Publié: (2025)

Fast Inference via Hierarchical Speculative Decoding
par: Mohri, Clara, et autres
Publié: (2025)

AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
par: Song, Dinghong, et autres
Publié: (2025)

Accelerating Transformer Inference for Translation via Parallel Decoding
par: Santilli, Andrea, et autres
Publié: (2023)

Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
par: Zhou, Xuwen, et autres
Publié: (2026)

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
par: Agrawal, Sudhanshu, et autres
Publié: (2025)

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
par: Xiao, Zilin, et autres
Publié: (2024)

MineDraft: A Framework for Batch Parallel Speculative Decoding
par: Tang, Zhenwei, et autres
Publié: (2026)

SRT: Accelerating Reinforcement Learning via Speculative Rollout with Tree-Structured Cache
par: Chang, Chi-Chih, et autres
Publié: (2026)

Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
par: Sun, Shuoyang, et autres
Publié: (2026)

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
par: Cai, Tianle, et autres
Publié: (2024)

Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding
par: Park, Jihoon, et autres
Publié: (2025)

Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching
par: Dong, Yanhao, et autres
Publié: (2025)

Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
par: Zhao, Yilong, et autres
Publié: (2025)

SpecMoE: A Fast and Efficient Mixture-of-Experts Inference via Self-Assisted Speculative Decoding
par: Bang, Jehyeon, et autres
Publié: (2026)

LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
par: Yi, Jiawei, et autres
Publié: (2025)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
par: Li, Ruixiao, et autres
Publié: (2025)

TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
par: Gong, Shukai, et autres
Publié: (2025)

Speculative Speculative Decoding
par: Kumar, Tanishq, et autres
Publié: (2026)

FastEagle: Cascaded Drafting for Accelerating Speculative Decoding
par: Huang, Haiduo, et autres
Publié: (2025)

Accelerating Time Series Foundation Models with Speculative Decoding
par: Subbaraman, Pranav, et autres
Publié: (2025)

AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
par: McDanel, Bradley
Publié: (2024)

QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
par: Tiwari, Rishabh, et autres
Publié: (2025)

Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling
par: Hao, Yongchang, et autres
Publié: (2026)

ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference
par: Zhang, Qiuyang, et autres
Publié: (2026)

Lever: Speculative LLM Inference on Smartphones
par: Wang, Tuowei, et autres
Publié: (2026)

Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
par: Yi, Hanling, et autres
Publié: (2024)

CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
par: Ning, Zhiyuan, et autres
Publié: (2025)

CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
par: Han, Yuning, et autres
Publié: (2026)

Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
par: Iso, Hayate, et autres
Publié: (2026)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)