Copertina

Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Siavashi, Mohammad, Scazzariello, Mariano, Maguire Jr., Gerald Q., Kostić, Dejan, Chiesa, Marco
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning Operating Systems Performance Software Engineering
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

CPU-Limits kill Performance: Time to rethink Resource Control
di: Shetty, Chirag, et al.
Pubblicazione: (2025)

Optimizing CPU Cache Utilization in Cloud VMs with Accurate Cache Abstraction
di: Tofigh, Mani, et al.
Pubblicazione: (2025)

GPUVM: GPU-driven Unified Virtual Memory
di: Nazaraliyev, Nurlan, et al.
Pubblicazione: (2024)

Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference
di: Siavashi, Mohammad, et al.
Pubblicazione: (2025)

HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing
di: Lin, Mao, et al.
Pubblicazione: (2026)

Mitigating GIL Bottlenecks in Edge AI Systems
di: Mandal, Mridankan, et al.
Pubblicazione: (2026)

RAID Organizations for Improved Reliability and Performance: A Not Entirely Unbiased Tutorial (1st revision)
di: Thomasian, Alexander
Pubblicazione: (2024)

SwitchFS: Asynchronous Metadata Updates for Distributed Filesystems with In-Network Coordination
di: Xu, Jingwei, et al.
Pubblicazione: (2024)

Demystifying Serverless Costs on Public Platforms: Bridging Billing, Architecture, and OS Scheduling
di: Lin, Changyuan, et al.
Pubblicazione: (2025)

Accelerating Mobile Inference through Fine-Grained CPU-GPU Co-Execution
di: Li, Zhuojin, et al.
Pubblicazione: (2025)

Dissecting CPU-GPU Unified Physical Memory on AMD MI300A APUs
di: Wahlgren, Jacob, et al.
Pubblicazione: (2025)

Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
di: Kamahori, Keisuke, et al.
Pubblicazione: (2024)

Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architectures
di: Vellaisamy, Prabhu, et al.
Pubblicazione: (2025)

Comparing CPU and GPU compute of PERMANOVA on MI300A
di: Sfiligoi, Igor
Pubblicazione: (2025)

Random Adaptive Cache Placement Policy
di: Ahire, Vrushank, et al.
Pubblicazione: (2025)

The Hitchhiker's Guide to Programming and Optimizing Cache Coherent Heterogeneous Systems: CXL, NVLink-C2C, and AMD Infinity Fabric
di: Wang, Zixuan, et al.
Pubblicazione: (2024)

Communication Offloading on SmartNIC DPUs: A Quantitative Approach
di: Wahlgren, Jacob, et al.
Pubblicazione: (2026)

Terabyte-Scale Analytics in the Blink of an Eye
di: Wu, Bowen, et al.
Pubblicazione: (2025)

OSMOSIS: Enabling Multi-Tenancy in Datacenter SmartNICs
di: Khalilov, Mikhail, et al.
Pubblicazione: (2023)

Optimizing Task Scheduling in Heterogeneous Computing Environments: A Comparative Analysis of CPU, GPU, and ASIC Platforms Using E2C Simulator
di: Mohammadjafari, Ali, et al.
Pubblicazione: (2024)

Plug & Offload: Transparently Offloading TCP Stack onto Off-path SmartNIC with PnO-TCP
di: Nan, Hailong, et al.
Pubblicazione: (2025)

Peformance Isolation for Inference Processes in Edge GPU Systems
di: Martín, Juan José, et al.
Pubblicazione: (2026)

A Case for CATS: A Conductor-driven Asymmetric Transport Scheme for Semantic Prioritization
di: Rizvi, Syed Muhammad Aqdas
Pubblicazione: (2026)

Disaggregated Memory with SmartNIC Offloading: a Case Study on Graph Processing
di: Wahlgren, Jacob, et al.
Pubblicazione: (2024)

Meili: Enabling SmartNIC as a Service in the Cloud
di: Su, Qiang, et al.
Pubblicazione: (2023)

Evaluating HPC-Style CPU Performance and Cost in Virtualized Cloud Infrastructures
di: Tharwani, Jay, et al.
Pubblicazione: (2025)

xMem: A CPU-Based Approach for Accurate Estimation of GPU Memory in Deep Learning Training Workloads
di: Shi, Jiabo, et al.
Pubblicazione: (2025)

Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
di: Zhuang, Chen, et al.
Pubblicazione: (2024)

Dissecting the software-based measurement of CPU energy consumption: a comparative analysis
di: Raffin, Guillaume, et al.
Pubblicazione: (2024)

SCENIC: Stream Computation-Enhanced SmartNIC
di: Ramhorst, Benjamin, et al.
Pubblicazione: (2026)

Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments
di: Iliakopoulou, Nikoleta, et al.
Pubblicazione: (2024)

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving
di: Sun, Tingyang, et al.
Pubblicazione: (2026)

FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
di: He, Jiaao, et al.
Pubblicazione: (2024)

The Energy Cost of Execution-Idle in GPU Clusters
di: Lei, Yiran, et al.
Pubblicazione: (2026)

Scalable GPU Performance Variability Analysis framework
di: Lahiry, Ankur, et al.
Pubblicazione: (2025)

On the Partitioning of GPU Power among Multi-Instances
di: Vamja, Tirth, et al.
Pubblicazione: (2025)

Disaggregated Design for GPU-Based Volumetric Data Structures
di: Meneghin, Massimiliano, et al.
Pubblicazione: (2025)

Taking GPU Programming Models to Task for Performance Portability
di: Davis, Joshua H., et al.
Pubblicazione: (2024)

Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors, Three Backends, and Three Browsers
di: Maczan, Jędrzej
Pubblicazione: (2026)

Towards Efficient and Practical GPU Multitasking in the Era of LLM
di: Xing, Jiarong, et al.
Pubblicazione: (2025)