Cover Image

Inference Performance Optimization for Large Language Models on CPUs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Pujiang, Zhou, Shan, Huang, Wenhuan, Li, Changqing, Wang, Duyi, Guo, Bin, Meng, Chen, Gui, Sheng, Yu, Weifei, Xie, Yi
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Distributed Inference Performance Optimization for LLMs on CPUs
by: He, Pujiang, et al.
Published: (2024)

Inference Acceleration for Large Language Models on CPUs
by: PS, Ditto, et al.
Published: (2024)

Sum of two squares in cyclic quartic fields
by: Huang, Wenhuan
Published: (2023)

A full-rank subgroup of Bloch Groups of CM Fields
by: Huang, Wenhuan
Published: (2025)

Sum of two squares in biquadratic fields
by: Huang, Wenhuan
Published: (2024)

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
by: Zhang, Haolin, et al.
Published: (2025)

Highly Optimized Kernels and Fine-Grained Codebooks for LLM Inference on Arm CPUs
by: Gope, Dibakar, et al.
Published: (2024)

Fast and Compact Tsetlin Machine Inference on CPUs Using Instruction-Level Optimization
by: Zeng, Yefan, et al.
Published: (2025)

Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache
by: Yu, Ye, et al.
Published: (2026)

Enhanced Diagnostic Performance via Large-Resolution Inference Optimization for Pathology Foundation Models
by: Hu, Mengxuan, et al.
Published: (2026)

SLO-Aware Scheduling for Large Language Model Inferences
by: Huang, Jinqi, et al.
Published: (2025)

Dynamic Network Risk Identification of Moving Targets in Flight Area Based on Community State
by: Duyi Zhou, et al.
Published: (2026)

Computational Reasoning of Large Language Models
by: Wu, Haitao, et al.
Published: (2025)

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
by: Zhang, Xinyuan, et al.
Published: (2024)

IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs
by: Mao, Yuzhen, et al.
Published: (2024)

Are Large Language Models Good Prompt Optimizers?
by: Ma, Ruotian, et al.
Published: (2024)

NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
by: Zhang, Tianyi, et al.
Published: (2024)

TIGFlow-GRPO: Trajectory Forecasting via Interaction-Aware Flow Matching and Reward-Guided Optimization
by: Jing, Xuepeng, et al.
Published: (2026)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
by: Dai, Jincheng, et al.
Published: (2024)

Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
by: Li, Jinhao, et al.
Published: (2024)

HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
by: Zhao, Feiyu, et al.
Published: (2026)

TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
by: Chen, Junzhe, et al.
Published: (2026)

When Large Language Model Meets Optimization
by: Huang, Sen, et al.
Published: (2024)

Membership Inference Attacks on Tokenizers of Large Language Models
by: Teng, Meng
Published: (2025)

Litespark Inference on Consumer CPUs: Custom SIMD Kernels for Ternary Neural Networks
by: Dade, Nii Osae Osae, et al.
Published: (2026)

ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs
by: Xu, Yuzhuang, et al.
Published: (2026)

Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference
by: Chen, Chun-Ting, et al.
Published: (2025)

Optimizing Resource Allocation for Geographically-Distributed Inference by Large Language Models
by: Sun, Tingyang, et al.
Published: (2025)

Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy
by: Liu, Yijin, et al.
Published: (2024)

DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Models
by: Zhou, Xirui, et al.
Published: (2025)

Accelerating Graph Indexing for ANNS on Modern CPUs
by: Wang, Mengzhao, et al.
Published: (2025)

Bridging Code Graphs and Large Language Models for Better Code Understanding
by: Chen, Zeqi, et al.
Published: (2025)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Pressure‐Induced Pre‐Lithiation Enables High‐Performing Si Anodes in All‐Solid‐State Batteries
by: Weifei Hu, et al.
Published: (2024)

Crystal Structure, Band Gap, and Optoelectronic Performance of Lead‐Free Cs2FeCl5⋅H2O Erythrosiderite‐Halide Single Crystal and Its Thin Film
by: Chen Wang, et al.
Published: (2025)

Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
by: Sui, Yuan, et al.
Published: (2025)

AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization
by: Liu, Chaohu, et al.
Published: (2025)

Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference
by: Huang, Yafan, et al.
Published: (2026)

Towards High-Performance and Portable Molecular Docking on CPUs through Vectorization
by: Accordi, Gianmarco, et al.
Published: (2025)

Membership Inference Attacks on Tokenizers of Large Language Models
by: Tong, Meng, et al.
Published: (2025)