Copertina

SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wu, Hang, Zhu, Jianian, Li, Yinghui, Wang, Haojie, Hou, Biao, Zhai, Jidong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
di: Zhu, Jianian, et al.
Pubblicazione: (2025)

FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
di: Li, Yuchen, et al.
Pubblicazione: (2026)

SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
di: Xu, Jiaming, et al.
Pubblicazione: (2025)

NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
di: Huang, Jiacheng, et al.
Pubblicazione: (2025)

Accelerating OpenPangu Inference on NPU via Speculative Decoding
di: Dai, Yuntao, et al.
Pubblicazione: (2026)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
di: Li, Rui, et al.
Pubblicazione: (2025)

SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
di: Zhang, Ziyi, et al.
Pubblicazione: (2025)

ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
di: Chen, Qiaoling, et al.
Pubblicazione: (2025)

SpecMemo: Speculative Decoding is in Your Pocket
di: Yildirim, Selin, et al.
Pubblicazione: (2025)

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
di: Zheng, Ce, et al.
Pubblicazione: (2026)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
di: Liu, Xing, et al.
Pubblicazione: (2025)

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
di: Shen, Yuhao, et al.
Pubblicazione: (2025)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
di: Tran, Phuong, et al.
Pubblicazione: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
di: Chen, Fahao, et al.
Pubblicazione: (2025)

SpecInF: Exploiting Idle GPU Resources in Distributed DL Training via Speculative Inference Filling
di: Lv, Cunchi, et al.
Pubblicazione: (2025)

Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
di: Wang, Siqi, et al.
Pubblicazione: (2024)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
di: Chen, Xing, et al.
Pubblicazione: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
di: Zheng, Size, et al.
Pubblicazione: (2026)

SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection
di: Shukla, Shikhar
Pubblicazione: (2026)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
di: Wei, Jinhui, et al.
Pubblicazione: (2025)

FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
di: Chen, Wenyan, et al.
Pubblicazione: (2026)

Tasking framework for Adaptive Speculative Parallel Mesh Generation
di: Tsolakis, Christos, et al.
Pubblicazione: (2024)

NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
di: Chen, Jiefei, et al.
Pubblicazione: (2026)

A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
di: Zhang, Yida, et al.
Pubblicazione: (2026)

PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding
di: Han, Yunhe, et al.
Pubblicazione: (2026)

RouterWise: Joint Resource Allocation and Routing for Latency-Aware Multi-Model LLM Serving
di: Kasnavieh, Hossein Hosseini, et al.
Pubblicazione: (2026)

DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
di: Yu, Fengze, et al.
Pubblicazione: (2025)

PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
di: McDanel, Bradley, et al.
Pubblicazione: (2025)

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
di: Xiang, Yuxing, et al.
Pubblicazione: (2025)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
di: Jiang, Lijuan, et al.
Pubblicazione: (2025)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
di: Wang, Wenfeng, et al.
Pubblicazione: (2025)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
di: Chen, Shaoyuan, et al.
Pubblicazione: (2024)

SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
di: Abramovich, Talor, et al.
Pubblicazione: (2026)

ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
di: Tang, Ding, et al.
Pubblicazione: (2024)

PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption
di: Tan, Yifan, et al.
Pubblicazione: (2024)