Copertina

Autellix: An Efficient Serving Engine for LLM Agents as General Programs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Luo, Michael, Shi, Xiaoxiang, Cai, Colin, Zhang, Tianjun, Wong, Justin, Wang, Yichuan, Wang, Chi, Huang, Yanping, Chen, Zhifeng, Gonzalez, Joseph E., Stoica, Ion
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
di: Duan, Jiangfei, et al.
Pubblicazione: (2024)

Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
di: Shi, Xiaoxiang, et al.
Pubblicazione: (2025)

Locality-aware Fair Scheduling in LLM Serving
di: Cao, Shiyi, et al.
Pubblicazione: (2025)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
di: Qiao, Yifan, et al.
Pubblicazione: (2024)

Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
di: Liu, Xueshen, et al.
Pubblicazione: (2026)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
di: Griggs, Tyler, et al.
Pubblicazione: (2024)

TENT: A Declarative Slice Spraying Engine for Performant and Resilient Data Movement in Disaggregated LLM Serving
di: Ren, Feng, et al.
Pubblicazione: (2026)

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
di: Du, Kuntai, et al.
Pubblicazione: (2025)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
di: Xu, Jiale, et al.
Pubblicazione: (2025)

T-RBFT: A Scalable and Efficient Byzantine Consensus Based on Trusted Execution Environment for Consortium Blockchain
di: Gao, Wen, et al.
Pubblicazione: (2026)

Towards Efficient and Practical GPU Multitasking in the Era of LLM
di: Xing, Jiarong, et al.
Pubblicazione: (2025)

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
di: Mao, Ziming, et al.
Pubblicazione: (2024)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
di: Hu, Cunchen, et al.
Pubblicazione: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Pie: Pooling CPU Memory for LLM Inference
di: Xu, Yi, et al.
Pubblicazione: (2024)

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
di: Ye, Zihao, et al.
Pubblicazione: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
di: Peng, You, et al.
Pubblicazione: (2026)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
di: He, Yiyuan, et al.
Pubblicazione: (2024)

Efficient Multi-round LLM Inference over Disaggregated Serving
di: He, Wenhao, et al.
Pubblicazione: (2026)

SkyWalker: A Locality-Aware Cross-Region Load Balancer for LLM Inference
di: Xia, Tian, et al.
Pubblicazione: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
di: Du, Boxiao, et al.
Pubblicazione: (2026)

AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
di: Zhang, Yuning, et al.
Pubblicazione: (2026)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
di: Yu, Shan, et al.
Pubblicazione: (2025)

DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling
di: Ahmad, Sohaib, et al.
Pubblicazione: (2024)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
di: Li, Suyi, et al.
Pubblicazione: (2024)

WWW.Serve: Interconnecting Global LLM Services through Decentralization
di: Wang, Huanyu, et al.
Pubblicazione: (2026)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
di: Jaiswal, Shashwat, et al.
Pubblicazione: (2025)

OmniInfer: System-Wide Acceleration Techniques for Optimizing LLM Serving Throughput and Latency
di: Wang, Jun, et al.
Pubblicazione: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
di: Ruan, Chaoyi, et al.
Pubblicazione: (2025)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
di: He, Xuan, et al.
Pubblicazione: (2025)

Graph for Science: From API based Programming to Graph Engine based Programming for HPC
di: Zhang, Yu, et al.
Pubblicazione: (2023)

Communication-Efficient Serving for Video Diffusion Models with Latent Parallelism
di: Wu, Zhiyuan, et al.
Pubblicazione: (2025)

Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
di: Wang, Shaoyu, et al.
Pubblicazione: (2025)

CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands
di: Wang, Weiye, et al.
Pubblicazione: (2026)

Flash-KMeans: Fast and Memory-Efficient Exact K-Means
di: Yang, Shuo, et al.
Pubblicazione: (2026)

MoLink: Distributed and Efficient Serving Framework for Large Models
di: Jin, Lewei, et al.
Pubblicazione: (2025)

SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
di: Cheng, Ke, et al.
Pubblicazione: (2024)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
di: Shen, Haiying, et al.
Pubblicazione: (2024)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
di: Jiang, Youhe, et al.
Pubblicazione: (2026)