Cover Image

PowerInfer-2: Fast Large Language Model Inference on a Smartphone

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xue, Zhenliang, Song, Yixin, Mi, Zeyu, Zheng, Xinrui, Xia, Yubin, Chen, Haibo
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
by: Song, Yixin, et al.
Published: (2023)

SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment
by: Song, Yixin, et al.
Published: (2025)

DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
by: Xue, Zhenliang, et al.
Published: (2025)

SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses
by: Chen, Le, et al.
Published: (2026)

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
by: Song, Yixin, et al.
Published: (2024)

EvolveSignal: A Large Language Model Powered Coding Agent for Discovering Traffic Signal Control Strategies
by: Wang, Leizhen, et al.
Published: (2025)

CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activation
by: Wang, Qinsi, et al.
Published: (2024)

Fast Inference for Augmented Large Language Models
by: Shahout, Rana, et al.
Published: (2024)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
by: Miao, Xupeng, et al.
Published: (2023)

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
by: Abhyankar, Reyna, et al.
Published: (2024)

L3: DIMM-PIM Integrated Architecture and Coordination for Scalable Long-Context LLM Inference
by: Liu, Qingyuan, et al.
Published: (2025)

Lever: Speculative LLM Inference on Smartphones
by: Wang, Tuowei, et al.
Published: (2026)

Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing
by: Qian, Zhenyu, et al.
Published: (2024)

Large Language Model Enhanced Machine Learning Estimators for Classification
by: Wu, Yuhang, et al.
Published: (2024)

Fast Distributed Inference Serving for Large Language Models
by: Wu, Bingyang, et al.
Published: (2023)

Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
by: Chen, Le, et al.
Published: (2025)

Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linking
by: Wang, Tuowei, et al.
Published: (2024)

Inferring the Size of Large Language Models From Popular Text Memorization
by: Nikolic, Ivica
Published: (2026)

FastAV: Efficient Token Pruning for Audio-Visual Large Language Model Inference
by: Jung, Chaeyoung, et al.
Published: (2026)

Causal Inference for Human-Language Model Collaboration
by: Zhang, Bohan, et al.
Published: (2024)

Towards Multimodal Graph Large Language Model
by: Wang, Xin, et al.
Published: (2025)

FDC: Fast KV Dimensionality Compression for Efficient LLM Inference
by: Zhang, Zeyu, et al.
Published: (2024)

Local Prediction-Powered Inference
by: Gu, Yanwu, et al.
Published: (2024)

Assessing Adversarial Robustness of Large Language Models: An Empirical Study
by: Yang, Zeyu, et al.
Published: (2024)

FAS: Fast ANN-SNN Conversion for Spiking Large Language Models
by: Chen, Long, et al.
Published: (2025)

FASP: Fast and Accurate Structured Pruning of Large Language Models
by: Hu, Hanyu, et al.
Published: (2025)

LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Models
by: Peng, Zengqi, et al.
Published: (2025)

MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
by: Wang, Zhaode, et al.
Published: (2025)

GNNBleed: Inference Attacks to Unveil Private Edges in Graphs with Realistic Access to GNN Models
by: Song, Zeyu, et al.
Published: (2023)

PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption
by: Tan, Yifan, et al.
Published: (2024)

Can Large Language Models Infer Causation from Correlation?
by: Jin, Zhijing, et al.
Published: (2023)

SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
by: Wang, Xin, et al.
Published: (2024)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
by: Zhang, Zhengyan, et al.
Published: (2024)

Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation
by: Fisch, Adam, et al.
Published: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference
by: Bajpai, Divya Jyoti, et al.
Published: (2025)

Fast Inference with Kronecker-Sparse Matrices
by: Gonon, Antoine, et al.
Published: (2024)

Unlocking Emergent Modularity in Large Language Models
by: Qiu, Zihan, et al.
Published: (2023)

Large Language Models Meet Graph Neural Networks for Text-Numeric Graph Reasoning
by: Song, Haoran, et al.
Published: (2025)

Demystifying Prediction Powered Inference
by: Song, Yilin, et al.
Published: (2026)