Copertina

On the Cost of Model-Serving Frameworks: An Experimental Evaluation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	De Rosa, Pasquale, Bromberg, Yérom-David, Felber, Pascal, Mvondo, Djob, Schiavoni, Valerio
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Practical Forecasting of Cryptocoins Timeseries using Correlation Patterns
di: De Rosa, Pasquale, et al.
Pubblicazione: (2024)

PhishingHook: Catching Phishing Ethereum Smart Contracts leveraging EVM Opcodes
di: De Rosa, Pasquale, et al.
Pubblicazione: (2025)

ScamDetect: Towards a Robust, Agnostic Framework to Uncover Threats in Smart Contracts
di: De Rosa, Pasquale, et al.
Pubblicazione: (2025)

Plinius: Secure and Persistent Machine Learning Model Training
di: Yuhala, Peterson, et al.
Pubblicazione: (2021)

CryptoAnalytics: Cryptocoins Price Forecasting with Machine Learning Techniques
di: De Rosa, Pasquale, et al.
Pubblicazione: (2024)

BlindexTEE: A Blind Index Approach towards TEE-supported End-to-end Encrypted DBMS
di: Vialar, Louis, et al.
Pubblicazione: (2024)

IM-PIR: In-Memory Private Information Retrieval
di: Mwaisela, Mpoki, et al.
Pubblicazione: (2025)

PIM-CACHE: High-Efficiency Content-Aware Copy for Processing-In-Memory
di: Yuhala, Peterson, et al.
Pubblicazione: (2026)

Evaluating the Potential of In-Memory Processing to Accelerate Homomorphic Encryption
di: Mwaisela, Mpoki, et al.
Pubblicazione: (2024)

Continuous Semantic Caching for Low-Cost LLM Serving
di: Atalar, Baran, et al.
Pubblicazione: (2026)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
di: Gao, Bin, et al.
Pubblicazione: (2024)

Partition Detection in Byzantine Networks
di: Bromberg, Yérom-David, et al.
Pubblicazione: (2024)

Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
di: Liu, Xutong, et al.
Pubblicazione: (2025)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
di: Griggs, Tyler, et al.
Pubblicazione: (2024)

HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing
di: Xue, Leyang, et al.
Pubblicazione: (2025)

Cost-Optimal Active AI Model Evaluation
di: Angelopoulos, Anastasios N., et al.
Pubblicazione: (2025)

Model Equality Testing: Which Model Is This API Serving?
di: Gao, Irena, et al.
Pubblicazione: (2024)

A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention
di: Lee, Heejun, et al.
Pubblicazione: (2024)

TriHaRd: Higher Resilience for TEE Trusted Time
di: Bettinger, Matthieu, et al.
Pubblicazione: (2025)

Practical Secure Aggregation by Combining Cryptography and Trusted Execution Environments
di: de Laage, Romain, et al.
Pubblicazione: (2025)

CascadeServe: Unlocking Model Cascades for Inference Serving
di: Kossmann, Ferdi, et al.
Pubblicazione: (2024)

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
di: Zeng, Pai, et al.
Pubblicazione: (2024)

TorchAO: PyTorch-Native Training-to-Serving Model Optimization
di: Or, Andrew, et al.
Pubblicazione: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
di: Kim, Kihyun, et al.
Pubblicazione: (2025)

IC-Cache: Efficient Large Language Model Serving via In-context Caching
di: Yu, Yifan, et al.
Pubblicazione: (2025)

Defining 'Good': Evaluation Framework for Synthetic Smart Meter Data
di: Chai, Sheng, et al.
Pubblicazione: (2024)

DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
di: Gao, Lei, et al.
Pubblicazione: (2025)

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
di: Gao, Shihong, et al.
Pubblicazione: (2025)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
di: Zhao, Adrian, et al.
Pubblicazione: (2026)

EdgeServe: A Streaming System for Decentralized Model Serving
di: Shaowang, Ted, et al.
Pubblicazione: (2023)

iServe: An Intent-based Serving System for LLMs
di: Liakopoulos, Dimitrios, et al.
Pubblicazione: (2025)

An Interpretable Latency Model for Speculative Decoding in LLM Serving
di: Kong, Linghao, et al.
Pubblicazione: (2026)

FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
di: Wang, Minghe, et al.
Pubblicazione: (2026)

Conditional computation in neural networks: principles and research trends
di: Scardapane, Simone, et al.
Pubblicazione: (2024)

Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs
di: Wang, Ganghua, et al.
Pubblicazione: (2025)

Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules
di: Landsgesell, Jonas, et al.
Pubblicazione: (2026)

Fairness in Serving Large Language Models
di: Sheng, Ying, et al.
Pubblicazione: (2023)

A Framework for Bounding Deterministic Risk with PAC-Bayes: Applications to Majority Votes
di: Leblanc, Benjamin, et al.
Pubblicazione: (2025)

Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
di: Yu, Shan, et al.
Pubblicazione: (2025)

Instance-Level Costs for Nuanced Classifier Evaluation
di: Kang, Kabir, et al.
Pubblicazione: (2026)