Copertina

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Fahao, Li, Peng, Luan, Tom H., Su, Zhou, Deng, Jing
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
di: Wang, Siqi, et al.
Pubblicazione: (2024)

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices
di: Chen, Fahao, et al.
Pubblicazione: (2025)

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
di: Chen, Fahao, et al.
Pubblicazione: (2024)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
di: Qianli, Liu, et al.
Pubblicazione: (2025)

SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
di: Xu, Jiaming, et al.
Pubblicazione: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

Accelerating OpenPangu Inference on NPU via Speculative Decoding
di: Dai, Yuntao, et al.
Pubblicazione: (2026)

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment
di: Jiang, Youhe, et al.
Pubblicazione: (2023)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

Inference Acceleration for Large Language Models on CPUs
di: PS, Ditto, et al.
Pubblicazione: (2024)

SLO-Aware Scheduling for Large Language Model Inferences
di: Huang, Jinqi, et al.
Pubblicazione: (2025)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
di: Zhao, Xuanlei, et al.
Pubblicazione: (2024)

SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference
di: Zhao, Yihao, et al.
Pubblicazione: (2025)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
di: Yu, Minchen, et al.
Pubblicazione: (2025)

SLIM: A Heterogeneous Accelerator for Edge Inference of Sparse Large Language Model via Adaptive Thresholding
di: Xu, Weihong, et al.
Pubblicazione: (2025)

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
di: Ma, Chenxiang, et al.
Pubblicazione: (2025)

Bandwidth-Aware LLM Inference on Heterogeneous Many-Core Supercomputers
di: Lu, Yao, et al.
Pubblicazione: (2026)

Network Edge Inference for Large Language Models: Principles, Techniques, and Opportunities
di: Chen, Zhixiong, et al.
Pubblicazione: (2026)

Federated Inference for Heterogeneous LLM Communication and Collaboration
di: Chen, Zihan, et al.
Pubblicazione: (2026)

Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
di: Wei, Jinhui, et al.
Pubblicazione: (2025)

Characterizing Communication Patterns in Distributed Large Language Model Inference
di: Xu, Lang, et al.
Pubblicazione: (2025)

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
di: Zhang, Zili, et al.
Pubblicazione: (2024)

HarmonyBatch: Batching multi-SLO DNN Inference with Heterogeneous Serverless Functions
di: Chen, Jiabin, et al.
Pubblicazione: (2024)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
di: Chen, Xing, et al.
Pubblicazione: (2025)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
di: Li, Rui, et al.
Pubblicazione: (2025)

Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
di: He, Zifan, et al.
Pubblicazione: (2026)

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
di: Yan, Ran, et al.
Pubblicazione: (2024)

A Unified Programming Model for Heterogeneous Computing with CPU and Accelerator Technologies
di: Xiong, Yuqing
Pubblicazione: (2022)

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
di: Du, Kuntai, et al.
Pubblicazione: (2025)

A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
di: Zhang, Yida, et al.
Pubblicazione: (2026)

An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models
di: Hakim, Sheikh Azizul, et al.
Pubblicazione: (2025)

FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
di: QI, Jiaxing, et al.
Pubblicazione: (2024)

GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
di: Tran, Phuong, et al.
Pubblicazione: (2025)

PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption
di: Tan, Yifan, et al.
Pubblicazione: (2024)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
di: Wu, Tian, et al.
Pubblicazione: (2025)

PALM: A Efficient Performance Simulator for Tiled Accelerators with Large-scale Model Training
di: Fang, Jiahao, et al.
Pubblicazione: (2024)

TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems
di: Wu, Feiyang, et al.
Pubblicazione: (2025)

SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
di: Wu, Hang, et al.
Pubblicazione: (2025)

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
di: Zheng, Ce, et al.
Pubblicazione: (2026)