Copertina

Fairness in Serving Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sheng, Ying, Cao, Shiyi, Li, Dacheng, Zhu, Banghua, Li, Zhuohan, Zhuo, Danyang, Gonzalez, Joseph E., Stoica, Ion
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Artificial Intelligence Machine Learning Performance
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput
di: Liu, Xiaoxuan, et al.
Pubblicazione: (2024)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
di: Yu, Shan, et al.
Pubblicazione: (2025)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
di: Sheng, Ying, et al.
Pubblicazione: (2023)

ALISE: Accelerating Large Language Model Serving with Speculative Scheduling
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
di: Li, Jiaxi, et al.
Pubblicazione: (2025)

K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
di: Cao, Shiyi, et al.
Pubblicazione: (2026)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
di: Shao, Zishan, et al.
Pubblicazione: (2025)

FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
di: Wu, Wenhao, et al.
Pubblicazione: (2026)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
di: Zhang, Hang, et al.
Pubblicazione: (2025)

Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
di: Miao, Xupeng, et al.
Pubblicazione: (2023)

S*: Test Time Scaling for Code Generation
di: Li, Dacheng, et al.
Pubblicazione: (2025)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
di: Suo, Jiashun, et al.
Pubblicazione: (2025)

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
di: Chiang, Wei-Lin, et al.
Pubblicazione: (2024)

Data Efficacy for Language Model Training
di: Dai, Yalun, et al.
Pubblicazione: (2025)

Generalizing Scaling Laws for Dense and Sparse Large Language Models
di: Hossain, Md Arafat, et al.
Pubblicazione: (2025)

A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
di: Agullo, Ferran, et al.
Pubblicazione: (2025)

Deploying Open-Source Large Language Models: A performance Analysis
di: Bendi-Ouis, Yannis, et al.
Pubblicazione: (2024)

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
di: Jayakody, Shakya, et al.
Pubblicazione: (2026)

LOOPRAG: Enhancing Loop Transformation Optimization with Retrieval-Augmented Large Language Models
di: Zhi, Yijie, et al.
Pubblicazione: (2025)

SemaTune: Semantic-Aware Online OS Tuning with Large Language Models
di: Liargkovas, Georgios, et al.
Pubblicazione: (2026)

Breaking the Loop: Detecting and Mitigating Denial-of-Service Vulnerabilities in Large Language Models
di: Yu, Junzhe, et al.
Pubblicazione: (2025)

Edge Deployment of Small Language Models, a comprehensive comparison of CPU, GPU and NPU backends
di: Prieto, Pablo, et al.
Pubblicazione: (2025)

CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices
di: Paramanayakam, Varatheepan, et al.
Pubblicazione: (2025)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
di: Zheng, Wenhao, et al.
Pubblicazione: (2025)

Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems
di: Zhao, Yushang, et al.
Pubblicazione: (2025)

Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Models
di: Rosas, Miguel Romero, et al.
Pubblicazione: (2024)

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
di: Ma, Jeffrey Jian, et al.
Pubblicazione: (2025)

Faster LLM Inference using DBMS-Inspired Preemption and Cache Replacement Policies
di: Kim, Kyoungmin, et al.
Pubblicazione: (2024)

Model Compression and Efficient Inference for Large Language Models: A Survey
di: Wang, Wenxiao, et al.
Pubblicazione: (2024)

Can Large Language Models Predict Parallel Code Performance?
di: Bolet, Gregory, et al.
Pubblicazione: (2025)

DeepContext: A Context-aware, Cross-platform, and Cross-framework Tool for Performance Profiling and Analysis of Deep Learning Workloads
di: Zhao, Qidong, et al.
Pubblicazione: (2024)

SweetSpot: An Analytical Model for Predicting Energy Efficiency of LLM Inference
di: Cavagna, Hiari Pizzini, et al.
Pubblicazione: (2026)

Root Cause Localization for Microservice Systems in Cloud-edge Collaborative Environments
di: Zhu, Yuhan, et al.
Pubblicazione: (2024)

On the Compression of Language Models for Code: An Empirical Study on CodeBERT
di: d'Aloisio, Giordano, et al.
Pubblicazione: (2024)

Personalized Model-Based Design of Human Centric AI enabled CPS for Long term usage
di: Ngabonziza, Bernard, et al.
Pubblicazione: (2026)

Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations
di: Tyukin, Georgy
Pubblicazione: (2024)

SGLang: Efficient Execution of Structured Language Model Programs
di: Zheng, Lianmin, et al.
Pubblicazione: (2023)

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
di: Lin, Yujun, et al.
Pubblicazione: (2024)

Computable Fairness: Boltzmann-Softmax Control for AI Resource Allocation
di: Park, Ji-Won, et al.
Pubblicazione: (2026)