Cover Image

SGLang: Efficient Execution of Structured Language Model Programs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Lianmin, Yin, Liangsheng, Xie, Zhiqiang, Sun, Chuyue, Huang, Jeff, Yu, Cody Hao, Cao, Shiyi, Kozyrakis, Christos, Stoica, Ion, Gonzalez, Joseph E., Barrett, Clark, Sheng, Ying
Format:	Preprint
Published:	2023
Subjects:	Artificial Intelligence Programming Languages
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Execution
by: Xie, Zhiqiang, et al.
Published: (2024)

Post-Training Sparse Attention with Double Sparsity
by: Yang, Shuo, et al.
Published: (2024)

Clover: Closed-Loop Verifiable Code Generation
by: Sun, Chuyue, et al.
Published: (2023)

K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
by: Cao, Shiyi, et al.
Published: (2026)

Teaching Cloud Infrastructure and Scalable Application Deployment in an Undergraduate Computer Science Program
by: Saligrama, Aditya, et al.
Published: (2024)

FailSafe: High-performance Resilient Serving
by: Xu, Ziyi, et al.
Published: (2025)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

Locality-aware Fair Scheduling in LLM Serving
by: Cao, Shiyi, et al.
Published: (2025)

Fairness in Serving Large Language Models
by: Sheng, Ying, et al.
Published: (2023)

Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight
by: Xie, Zhiqiang, et al.
Published: (2024)

Efficient GNN Training Through Structure-Aware Randomized Mini-Batching
by: Balaji, Vignesh, et al.
Published: (2025)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
by: Jiang, Xuanlin, et al.
Published: (2024)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
by: Yu, Shan, et al.
Published: (2025)

Sparse Checkpointing for Fast and Reliable MoE Training
by: Gandhi, Swapnil, et al.
Published: (2024)

Some Present-Day Problems of Romanian Library Science
by: Stoica, Ion
Published: (1973)

The Central University Library, Bucharest. Over Seventy-five Years in the History of a Collection
by: Stoica, Ion
Published: (1972)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
by: Park, Jongseok, et al.
Published: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
by: Cao, Shiyi, et al.
Published: (2024)

Executive individualism and the tone of firms' annual reports
by: Wei Jiang, et al.
Published: (2024)

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
by: Chiang, Wei-Lin, et al.
Published: (2024)

On Optimizing the Communication of Model Parallelism
by: Zhuang, Yonghao, et al.
Published: (2022)

Pantograph: A Machine-to-Machine Interaction Interface for Advanced Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4
by: Aniva, Leni, et al.
Published: (2024)

cedar: Optimized and Unified Machine Learning Input Data Pipelines
by: Zhao, Mark, et al.
Published: (2024)

Hunting CUDA Bugs at Scale with cuFuzz
by: ziad, Mohamed Tarek Ibn, et al.
Published: (2026)

BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching
by: Zhao, Yilong, et al.
Published: (2024)

S*: Test Time Scaling for Code Generation
by: Li, Dacheng, et al.
Published: (2025)

The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
by: Luan, Frank Sifei, et al.
Published: (2025)

LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
by: Zheng, Lianmin, et al.
Published: (2023)

VeriStruct: AI-assisted Automated Verification of Data-Structure Modules in Verus
by: Sun, Chuyue, et al.
Published: (2025)

Strata: Hierarchical Context Caching for Long Context Language Model Serving
by: Xie, Zhiqiang, et al.
Published: (2025)

Efficient LLM Scheduling by Learning to Rank
by: Fu, Yichao, et al.
Published: (2024)

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
by: Jiang, Wenqi, et al.
Published: (2026)

LIMINAL: Exploring The Frontiers of LLM Decode Performance
by: Davies, Michael, et al.
Published: (2025)

ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation
by: Gandhi, Swapnil, et al.
Published: (2024)

Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
by: Fu, Yichao, et al.
Published: (2024)

SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling
by: Skiadopoulos, Athinagoras, et al.
Published: (2025)

LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
by: Li, Dacheng, et al.
Published: (2025)

Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile
by: Ding, Hangliang, et al.
Published: (2025)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
by: Luo, Michael, et al.
Published: (2025)

Regulating Branch Parallelism in LLM Serving
by: Gandhi, Swapnil, et al.
Published: (2026)