Copertina

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Oliaro, Gabriele, Miao, Xupeng, Cheng, Xinhao, Kada, Vineeth, Wu, Mengdi, Gao, Ruohan, Huang, Yingyi, Delacourt, Remi, Yang, April, Wang, Yingcheng, Unger, Colin, Jia, Zhihao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
di: Li, Zikun, et al.
Pubblicazione: (2025)

Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
di: Shen, Haiying, et al.
Pubblicazione: (2024)

FlexLLM: Exploring LLM Customization for Moving Target Defense on Black-Box LLMs Against Jailbreak Attacks
di: Chen, Bocheng, et al.
Pubblicazione: (2024)

Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
di: Mo, Zizhao, et al.
Pubblicazione: (2026)

AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applications
di: Shen, Haiying, et al.
Pubblicazione: (2025)

Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
di: Mei, Yixuan, et al.
Pubblicazione: (2026)

Aladdin: Joint Placement and Scaling for SLO-Aware LLM Serving
di: Nie, Chengyi, et al.
Pubblicazione: (2024)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

FlexLLM: Composable HLS Library for Flexible Hybrid LLM Accelerator Design
di: Zhang, Jiahao, et al.
Pubblicazione: (2026)

SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving
di: Lysenstøen, Christian
Pubblicazione: (2026)

JITServe: SLO-aware LLM Serving with Imprecise Request Information
di: Zhang, Wei, et al.
Pubblicazione: (2025)

CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
di: Huang, Shaoyuan, et al.
Pubblicazione: (2026)

MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment
di: Li, Yufei, et al.
Pubblicazione: (2025)

SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications
di: Oliaro, Gabriele, et al.
Pubblicazione: (2024)

SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving
di: Kakolyris, Andreas Kosmas, et al.
Pubblicazione: (2024)

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
di: Shi, Xiaoxiang, et al.
Pubblicazione: (2025)

Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution
di: Xu, Yechen, et al.
Pubblicazione: (2024)

PolyServe: Efficient Multi-SLO Serving at Scale
di: Zhu, Kan, et al.
Pubblicazione: (2025)

SLOs-Serve: Optimized Serving of Multi-SLO LLMs
di: Chen, Siyuan, et al.
Pubblicazione: (2025)

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference
di: Du, Yin, et al.
Pubblicazione: (2026)

Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
di: Xu, Ao, et al.
Pubblicazione: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
di: Zhang, Biao, et al.
Pubblicazione: (2024)

Past-Future Scheduler for LLM Serving under SLA Guarantees
di: Gong, Ruihao, et al.
Pubblicazione: (2025)

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
di: Lin, Yanying, et al.
Pubblicazione: (2025)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
di: Lai, Ruiqi, et al.
Pubblicazione: (2025)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
di: Cheng, Ke, et al.
Pubblicazione: (2024)

Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
di: Mei, Yixuan, et al.
Pubblicazione: (2024)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
di: Guo, Tianyu, et al.
Pubblicazione: (2025)

Where does an LLM begin computing an instruction?
di: Pola, Aditya, et al.
Pubblicazione: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
di: Hu, Jianmin, et al.
Pubblicazione: (2025)

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving
di: Liu, Xunzhuo, et al.
Pubblicazione: (2026)

Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference
di: Wang, Qipeng
Pubblicazione: (2026)

SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
di: Chow, Will
Pubblicazione: (2025)

PromptTuner: SLO-Aware Elastic System for LLM Prompt Tuning
di: Gao, Wei, et al.
Pubblicazione: (2026)

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
di: Ma, Chenxiang, et al.
Pubblicazione: (2025)

CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question Answering
di: Yang, Hao, et al.
Pubblicazione: (2026)

Learning Dynamics of LLM Finetuning
di: Ren, Yi, et al.
Pubblicazione: (2024)

AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving
di: Huang, Kaiyu, et al.
Pubblicazione: (2025)