Image de couverture de livre

MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Zhaode, Yang, Jingbang, Qian, Xinyu, Xing, Shiwen, Jiang, Xiaotang, Lv, Chengfei, Zhang, Shengyu
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
par: Huang, Zhengxiang, et autres
Publié: (2025)

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
par: Li, Kunxi, et autres
Publié: (2025)

MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?
par: Zou, Xingze, et autres
Publié: (2026)

FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
par: Li, Kunxi, et autres
Publié: (2025)

PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
par: Jiang, Zhonghua, et autres
Publié: (2025)

RecGPT-Mobile: On-Device Large Language Models for User Intent Understanding in Taobao Feed Recommendation
par: Zhang, Bin, et autres
Publié: (2026)

AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization
par: Jiang, Zhonghua, et autres
Publié: (2025)

Fast Distributed Inference Serving for Large Language Models
par: Wu, Bingyang, et autres
Publié: (2023)

MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment
par: Huang, Hanxian, et autres
Publié: (2026)

Efficient Deployment of Large Language Models on Resource-constrained Devices
par: Yao, Zhiwei, et autres
Publié: (2025)

RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction
par: Liu, Sihao, et autres
Publié: (2026)

Large Language Models Inference Engines based on Spiking Neural Networks
par: Balaji, Adarsha, et autres
Publié: (2025)

ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation
par: Tang, Zihao, et autres
Publié: (2024)

Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation
par: Wang, Jiawei, et autres
Publié: (2024)

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
par: Zeng, Chao, et autres
Publié: (2024)

Efficient Deployment of Vision-Language Models on Mobile Devices: A Case Study on OnePlus 13R
par: Guerrero, Pablo Robin, et autres
Publié: (2025)

Fast Inference for Augmented Large Language Models
par: Shahout, Rana, et autres
Publié: (2024)

AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment
par: Fu, Yonggan, et autres
Publié: (2024)

EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
par: Sanyal, Arnab, et autres
Publié: (2025)

Scaling On-Device GPU Inference for Large Generative Models
par: Tang, Jiuqiang, et autres
Publié: (2025)

Fast NF4 Dequantization Kernels for Large Language Model Inference
par: Qi, Xiangbo, et autres
Publié: (2026)

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference
par: Chen, Yaoqi, et autres
Publié: (2025)

Resource-Efficient Generative AI Model Deployment in Mobile Edge Networks
par: Liang, Yuxin, et autres
Publié: (2024)

Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions
par: Niu, Chaoyue, et autres
Publié: (2025)

Fast and Compact Tsetlin Machine Inference on CPUs Using Instruction-Level Optimization
par: Zeng, Yefan, et autres
Publié: (2025)

Semantic Trimming and Auxiliary Multi-step Prediction for Generative Recommendation
par: Zhan, Tianyu, et autres
Publié: (2026)

Graph Neural Networks Automated Design and Deployment on Device-Edge Co-Inference Systems
par: Zhou, Ao, et autres
Publié: (2024)

Fast-PGM: Fast Probabilistic Graphical Model Learning and Inference
par: Jiang, Jiantong, et autres
Publié: (2024)

BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
par: Yang, Weiqin, et autres
Publié: (2026)

A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models
par: Xie, Zuan, et autres
Publié: (2025)

PLMM: Personal Large Language Models on Mobile Devices
par: Gong, Yuanhao
Publié: (2023)

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
par: Chitty-Venkata, Krishna Teja, et autres
Publié: (2024)

Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
par: Xiao, Jie, et autres
Publié: (2024)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
par: Hu, Xing, et autres
Publié: (2024)

Deploying Large AI Models on Resource-Limited Devices with Split Federated Learning
par: Qiang, Xianke, et autres
Publié: (2025)

WebLLM: A High-Performance In-Browser LLM Inference Engine
par: Ruan, Charlie F., et autres
Publié: (2024)

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
par: Zeng, Chao, et autres
Publié: (2024)

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices
par: Song, Congzheng, et autres
Publié: (2025)

PowerInfer-2: Fast Large Language Model Inference on a Smartphone
par: Xue, Zhenliang, et autres
Publié: (2024)

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
par: Liu, Zechun, et autres
Publié: (2024)