Imagen de Portada

SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Ziyi, Jiang, Ziheng, Jiang, Chengquan, Yu, Menghan, Zheng, Size, Lin, Haibin, Hoffmann, Henry, Liu, Xin
Formato:	Preprint
Publicado:	2025
Materias:	Distributed, Parallel, and Cluster Computing Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
por: Wang, Zhibin, et al.
Publicado: (2025)

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
por: Zheng, Ce, et al.
Publicado: (2026)

FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
por: Li, Yuchen, et al.
Publicado: (2026)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
por: Liu, Xing, et al.
Publicado: (2025)

Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
por: Wei, Jinhui, et al.
Publicado: (2025)

SpecMemo: Speculative Decoding is in Your Pocket
por: Yildirim, Selin, et al.
Publicado: (2025)

Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
por: Song, Jingwei, et al.
Publicado: (2025)

ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
por: Chen, Qiaoling, et al.
Publicado: (2025)

AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
por: McDanel, Bradley
Publicado: (2024)

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
por: Shen, Yuhao, et al.
Publicado: (2025)

SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
por: Wu, Hang, et al.
Publicado: (2025)

FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
por: Chen, Wenyan, et al.
Publicado: (2026)

PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
por: McDanel, Bradley, et al.
Publicado: (2025)

FLASH Viterbi: Fast and Adaptive Viterbi Decoding for Modern Data Systems
por: Deng, Ziheng, et al.
Publicado: (2025)

A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
por: Zhang, Yida, et al.
Publicado: (2026)

TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives
por: Zheng, Size, et al.
Publicado: (2025)

Revisiting Speculative Leaderless Protocols for Low-Latency BFT Replication
por: Qian, Daniel, et al.
Publicado: (2026)

Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
por: Zhang, Shulai, et al.
Publicado: (2025)

Accelerating OpenPangu Inference on NPU via Speculative Decoding
por: Dai, Yuntao, et al.
Publicado: (2026)

Striking the Right Balance between Compute and Copy: Improving LLM Inferencing Under Speculative Decoding
por: Ramachandran, Arun, et al.
Publicado: (2025)

Hyperion: Low-Latency Ultra-HD Video Analytics via Collaborative Vision Transformer Inference
por: Jiang, Linyi, et al.
Publicado: (2025)

Falcon: Advancing Asynchronous BFT Consensus for Lower Latency and Enhanced Throughput
por: Dai, Xiaohai, et al.
Publicado: (2025)

Asynchronous Latency and Fast Atomic Snapshot
por: Bezerra, João Paulo, et al.
Publicado: (2024)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
por: Kumar, Satyam, et al.
Publicado: (2026)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
por: Zheng, Size, et al.
Publicado: (2026)

GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
por: Tran, Phuong, et al.
Publicado: (2025)

PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding
por: Han, Yunhe, et al.
Publicado: (2026)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
por: Chen, Liangkun, et al.
Publicado: (2025)

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
por: Chang, Li-Wen, et al.
Publicado: (2024)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
por: Jin, Chao, et al.
Publicado: (2025)

Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
por: Jin, Tian, et al.
Publicado: (2025)

SpecInF: Exploiting Idle GPU Resources in Distributed DL Training via Speculative Inference Filling
por: Lv, Cunchi, et al.
Publicado: (2025)

SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
por: Abramovich, Talor, et al.
Publicado: (2026)

SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection
por: Shukla, Shikhar
Publicado: (2026)

Mahi-Mahi: Low-Latency Asynchronous BFT DAG-Based Consensus
por: Jovanovic, Philipp, et al.
Publicado: (2024)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
por: Li, Rui, et al.
Publicado: (2025)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
por: Xue, Chunyu, et al.
Publicado: (2026)

Chasing the Speed of Light: Low-Latency Planetary-Scale Adaptive Byzantine Consensus
por: Berger, Christian, et al.
Publicado: (2023)

Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
por: Hidayetoglu, Mert, et al.
Publicado: (2025)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
por: Li, Cong, et al.
Publicado: (2025)