Imagen de Portada

Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Benazir, Afsara, Lin, Felix Xiaozhu
Formato:	Preprint
Publicado:	2025
Materias:	Performance
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs
por: Benazir, Afsara, et al.
Publicado: (2026)

Profiling Apple Silicon Performance for ML Training
por: Feng, Dahua, et al.
Publicado: (2025)

Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Models
por: Benazir, Afsara, et al.
Publicado: (2025)

Speech Understanding on Tiny Devices with A Learning Cache
por: Benazir, Afsara, et al.
Publicado: (2023)

Accelerating Sparse Ternary GEMM for Quantized ML on Apple Silicon
por: Lipshitz, Baraq, et al.
Publicado: (2025)

Proto: A Guided Journey through Modern OS Construction
por: Choe, Wonkyo, et al.
Publicado: (2025)

When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon
por: Bergach, Mohamed Amine
Publicado: (2026)

RWKV-edge: Deeply Compressed RWKV for Resource-Constrained Devices
por: Choe, Wonkyo, et al.
Publicado: (2024)

Evaluation of Domain-Specific Architectures for General-Purpose Applications in Apple Silicon
por: López, Álvaro Corrochano, et al.
Publicado: (2025)

From 8 Seconds to 370ms: Kernel-Fused SAR Imaging on Apple Silicon via Single-Dispatch FFT Pipelines
por: Bergach, Mohamed Amine
Publicado: (2026)

Range, Not Precision: Block-Floating-Point Half-Precision FFT and SAR Imaging on Apple Silicon
por: Bergach, Mohamed Amine
Publicado: (2026)

GPU-Accelerated INT8 Quantization for KV Cache Compression in Large Language Models
por: Taneja, Maanas, et al.
Publicado: (2026)

Model Compression and Efficient Inference for Large Language Models: A Survey
por: Wang, Wenxiao, et al.
Publicado: (2024)

MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
por: Chitty-Venkata, Krishna Teja, et al.
Publicado: (2025)

Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Model
por: Chen, Mu-Chi, et al.
Publicado: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
por: Chu, Kexin, et al.
Publicado: (2025)

A Performance Evaluation of a Quantized Large Language Model on Various Smartphones
por: Çöplü, Tolga, et al.
Publicado: (2023)

Atys: An Efficient Profiling Framework for Identifying Hotspot Functions in Large-scale Cloud Microservices
por: Sun, Jiaqi, et al.
Publicado: (2025)

From Profiling to Optimization: Unveiling the Profile Guided Optimization
por: Liu, Bingxin, et al.
Publicado: (2025)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
por: Zhao, Youpeng, et al.
Publicado: (2024)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
por: Zhao, Xuanlei, et al.
Publicado: (2024)

Beating vDSP: A 138 GFLOPS Radix-8 Stockham FFT on Apple Silicon via Two-Tier Register-Threadgroup Memory Decomposition
por: Bergach, Mohamed Amine
Publicado: (2026)

Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
por: Liu, Songze, et al.
Publicado: (2025)

lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
por: Wang, Haoxin, et al.
Publicado: (2025)

Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
por: Javat, Abdurrahman, et al.
Publicado: (2026)

Statistical Modeling and Uncertainty Estimation of LLM Inference Systems
por: Ray, Kaustabha, et al.
Publicado: (2025)

Impact of Generative AI (Large Language Models) on the PRA model construction and maintenance, observations
por: Rychkov, Valentin, et al.
Publicado: (2024)

A Review on Proprietary Accelerators for Large Language Models
por: Park, Sihyeong, et al.
Publicado: (2025)

Large-Scale Data Parallelization of Product Quantization and Inverted Indexing Using Dask
por: Abraham, Ashley N., et al.
Publicado: (2026)

A Controlled Study of Memory Hierarchy Transitions in Quantum Circuit Simulation on Apple M4 Pro Unified Memory Architecture
por: Pratipat, Gyan
Publicado: (2026)

SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
por: Ma, Ming, et al.
Publicado: (2025)

Efficient Hybrid Amplitude-Phase Quantization for Multi-Antenna Relay System
por: Kim, Changdae, et al.
Publicado: (2025)

LFED: A Literary Fiction Evaluation Dataset for Large Language Models
por: Yu, Linhao, et al.
Publicado: (2024)

LoPace: A Lossless Optimized Prompt Accurate Compression Engine for Large Language Model Applications
por: Ulla, Aman
Publicado: (2026)

Static Reuse Profile Estimation for Array Applications
por: Razzak, Abdur, et al.
Publicado: (2024)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
por: Zheng, Wenhao, et al.
Publicado: (2025)

Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations
por: Tyukin, Georgy
Publicado: (2024)

Static Estimation of Reuse Profiles for Arrays in Nested Loops
por: Razzak, Abdur, et al.
Publicado: (2025)

SysLLMatic: Large Language Models are Software System Optimizers
por: Peng, Huiyun, et al.
Publicado: (2025)

ALISE: Accelerating Large Language Model Serving with Speculative Scheduling
por: Zhao, Youpeng, et al.
Publicado: (2024)