Imagen de Portada

Inference performance evaluation for LLMs on edge devices with a novel benchmarking framework and metric

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chen, Hao, Tian, Cong, He, Zixuan, Yu, Bin, Liu, Yepang, Cao, Jialun
Formato:	Preprint
Publicado:	2025
Materias:	Performance
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Reconsidering the performance of DEVS modeling and simulation environments using the DEVStone benchmark
por: Risco-Martín, José L., et al.
Publicado: (2024)

Anatomizing Deep Learning Inference in Web Browsers
por: Wang, Qipeng, et al.
Publicado: (2024)

denet, A lightweight command-line tool for process monitoring in benchmarking and beyond
por: Carrillo, Ben, et al.
Publicado: (2025)

Optimizing Stateful Microservice Migration in Kubernetes with MS2M and Forensic Checkpointing
por: Dinh-Tuan, Hai, et al.
Publicado: (2025)

A relação entre a «performance» social e a «performance» económico-financeira
por: Daniel Taborda
Publicado: (2007)

OmniSim: Simulating Hardware with C Speed and RTL Accuracy for High-Level Synthesis Designs
por: Sarkar, Rishov, et al.
Publicado: (2025)

Machine Learning-Guided Memory Optimization for DLRM Inference on Tiered Memory
por: Ren, Jie, et al.
Publicado: (2025)

The Price of Interoperability: Exploring Cross-Chain Bridges and Their Economic Consequences
por: Cao, Yiyue, et al.
Publicado: (2026)

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
por: Holmes, Connor, et al.
Publicado: (2024)

TALP-Pages: An easy-to-integrate continuous performance monitoring framework
por: Seitz, Valentin, et al.
Publicado: (2025)

LightningSimV2: Faster and Scalable Simulation for High-Level Synthesis via Graph Compilation and Optimization
por: Sarkar, Rishov, et al.
Publicado: (2024)

DeepContext: A Context-aware, Cross-platform, and Cross-framework Tool for Performance Profiling and Analysis of Deep Learning Workloads
por: Zhao, Qidong, et al.
Publicado: (2024)

Meta-Metrics and Best Practices for System-Level Inference Performance Benchmarking
por: Salaria, Shweta, et al.
Publicado: (2025)

Empirical validity of basic profile and conceptual evaluation model of the annual performance appraisal process of the National Colleges of Education lecturers in Sri Lanka
por: Pathiraja, N. Kumara
Publicado: (2025)

Redundant Array Computation Elimination
por: Wang, Zixuan, et al.
Publicado: (2025)

Dissecting Embedding Bag Performance in DLRM Inference
por: Ambati, Chandrish, et al.
Publicado: (2025)

State of the art of ergonomic costs as criterion for evaluating and improving organizational performance in industry
por: Silvana Duarte-dos Santos
Publicado: (2015)

Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
por: Arif, Moiz, et al.
Publicado: (2026)

Statistical Modeling and Uncertainty Estimation of LLM Inference Systems
por: Ray, Kaustabha, et al.
Publicado: (2025)

Root Cause Localization for Microservice Systems in Cloud-edge Collaborative Environments
por: Zhu, Yuhan, et al.
Publicado: (2024)

RWKV-edge: Deeply Compressed RWKV for Resource-Constrained Devices
por: Choe, Wonkyo, et al.
Publicado: (2024)

H2EAL: Hybrid-Bonding Architecture with Hybrid Sparse Attention for Efficient Long-Context LLM Inference
por: Fu, Zizhuo, et al.
Publicado: (2025)

Back to Bits: Extending Shannon's communication performance framework to computing
por: Hawkins, Max, et al.
Publicado: (2025)

Modeling Tradeoffs between mobility, cost, and performance in Edge Computing
por: Waseem, Muhammad Danish, et al.
Publicado: (2026)

Characterize LSM-tree Compaction Performance via On-Device LLM Inference
por: Ding, Jiabiao, et al.
Publicado: (2026)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
por: Zhao, Xuanlei, et al.
Publicado: (2024)

A dynamic parallel method for performance optimization on hybrid CPUs
por: Yu, Luo, et al.
Publicado: (2024)

Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
por: Benazir, Afsara, et al.
Publicado: (2025)

A high-performance and portable implementation of the SISSO method for CPUs and GPUs
por: Eibl, Sebastian, et al.
Publicado: (2025)

Explainable Port Mapping Inference with Sparse Performance Counters for AMD's Zen Architectures
por: Ritter, Fabian, et al.
Publicado: (2024)

SparseInfer: Training-free Prediction of Activation Sparsity for Fast LLM Inference
por: Shin, Jiho, et al.
Publicado: (2024)

Pulse-engineered Controlled-V gate and its applications on superconducting quantum device
por: Satoh, Takahiko, et al.
Publicado: (2021)

Opal: A Modular Framework for Optimizing Performance using Analytics and LLMs
por: Zaeed, Mohammad, et al.
Publicado: (2025)

Performance metrics for the continuous distribution of entanglement in multi-user quantum networks
por: Iñesta, Álvaro G., et al.
Publicado: (2023)

How Much Parallelism Is "Free"? A Principle of Near-Free Parallelism for Parallel Decoding
por: He, Minghua, et al.
Publicado: (2026)

LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind
por: Zhang, Li, et al.
Publicado: (2025)

Towards self-optimization of publish/subscribe IoT systems using continuous performance monitoring
por: Djahafi, Mohammed, et al.
Publicado: (2024)

Evaluating the impact of the L3 cache size of AMD EPYC CPUs on the performance of CFD applications
por: Lawenda, Marcin, et al.
Publicado: (2025)

Reducing Compute Waste in LLMs through Kernel-Level DVFS
por: Spaan, Jeffrey, et al.
Publicado: (2026)

Do LLMs Have Visualization Literacy? An Evaluation on Modified Visualizations to Test Generalization in Data Interpretation
por: Hong, Jiayi, et al.
Publicado: (2025)