Copertina

FairBatching: Fairness-Aware Batch Formation for LLM Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lyu, Hongtao, Liu, Boyue, Wu, Mingyu, Chen, Haibo
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
di: Zheng, Zhen, et al.
Pubblicazione: (2024)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
di: Zheng, Wanyi, et al.
Pubblicazione: (2025)

HarmonyBatch: Batching multi-SLO DNN Inference with Heterogeneous Serverless Functions
di: Chen, Jiabin, et al.
Pubblicazione: (2024)

Herring: Parallel Batch-Order-Fairness on DAG-based Blockchain Consensus
di: Putnik, Marko, et al.
Pubblicazione: (2026)

Constraint Programming Models For Serial Batch Scheduling With Minimum Batch Size
di: Huertas, Jorge A., et al.
Pubblicazione: (2025)

OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
di: Zheng, Yijie, et al.
Pubblicazione: (2025)

Equinox: Holistic Fair Scheduling in Serving Large Language Models
di: Wei, Zhixiang, et al.
Pubblicazione: (2025)

Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference
di: Chen, Huamin, et al.
Pubblicazione: (2026)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
di: Zhao, Bingzhe, et al.
Pubblicazione: (2025)

A Parallel CPU-GPU Framework for Batching Heuristic Operations in Depth-First Heuristic Search
di: Futuhi, Ehsan, et al.
Pubblicazione: (2025)

On Using Large-Batches in Federated Learning
di: Tyagi, Sahil
Pubblicazione: (2025)

Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks
di: Li, Haiyuan, et al.
Pubblicazione: (2026)

Justitia: Fair and Efficient Scheduling of Task-parallel LLM Agents with Selective Pampering
di: Yang, Mingyan, et al.
Pubblicazione: (2025)

CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
di: Chen, Qiaoling, et al.
Pubblicazione: (2026)

Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
di: Pang, Bowen, et al.
Pubblicazione: (2025)

TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
di: Stojkovic, Jovan, et al.
Pubblicazione: (2025)

Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
di: Kim, Joon Ha, et al.
Pubblicazione: (2026)

AntBatchInfer: Elastic Batch Inference in the Kubernetes Cluster
di: Li, Siyuan, et al.
Pubblicazione: (2024)

ProMoE: Fast MoE-based LLM Serving using Proactive Caching
di: Song, Xiaoniu, et al.
Pubblicazione: (2024)

Fairness-Aware Job Scheduling for Multi-Job Federated Learning
di: Shi, Yuxin, et al.
Pubblicazione: (2024)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
di: Wu, Linyu, et al.
Pubblicazione: (2025)

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
di: Argerich, Mauricio Fadel, et al.
Pubblicazione: (2026)

Joint Optimization of Offloading, Batching and DVFS for Multiuser Co-Inference
di: Xu, Yaodan, et al.
Pubblicazione: (2025)

KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving
di: Cheng, Rongxin, et al.
Pubblicazione: (2024)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
di: Liu, Xing, et al.
Pubblicazione: (2025)

MineDraft: A Framework for Batch Parallel Speculative Decoding
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

Design a Win-Win Strategy That Is Fair to Both Service Providers and Tasks When Rejection Is Not an Option
di: Trabelsi, Yohai, et al.
Pubblicazione: (2024)

GPU-Accelerated Batch-Dynamic Subgraph Matching
di: Qiu, Linshan, et al.
Pubblicazione: (2024)

Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning
di: Chen, Hongyao, et al.
Pubblicazione: (2025)

LAPS: A Length-Aware-Prefill LLM Serving System
di: She, Jianshu, et al.
Pubblicazione: (2026)

Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
di: Li, Rongzhi, et al.
Pubblicazione: (2025)

FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning
di: Wang, Zihui, et al.
Pubblicazione: (2024)

Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
di: Chen, Le, et al.
Pubblicazione: (2025)

Fast LLM Post-training via Decoupled and Fastest-of-N Speculation
di: Cheng, Rongxin, et al.
Pubblicazione: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
di: Liu, Dong, et al.
Pubblicazione: (2025)

Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
di: Song, Jingwei, et al.
Pubblicazione: (2025)

Accelerating LLM Inference with Precomputed Query Storage
di: Park, Jay H., et al.
Pubblicazione: (2025)

Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
di: Recasens, Pol G., et al.
Pubblicazione: (2025)

FedFair^3: Unlocking Threefold Fairness in Federated Learning
di: Javaherian, Simin, et al.
Pubblicazione: (2024)

GetBatch: Distributed Multi-Object Retrieval for ML Data Loading
di: Aizman, Alex, et al.
Pubblicazione: (2026)