Copertina

MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Zhaode, Yang, Jingbang, Qian, Xinyu, Xing, Shiwen, Jiang, Xiaotang, Lv, Chengfei, Zhang, Shengyu
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
di: Huang, Zhengxiang, et al.
Pubblicazione: (2025)

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
di: Li, Kunxi, et al.
Pubblicazione: (2025)

MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?
di: Zou, Xingze, et al.
Pubblicazione: (2026)

FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
di: Li, Kunxi, et al.
Pubblicazione: (2025)

PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
di: Jiang, Zhonghua, et al.
Pubblicazione: (2025)

RecGPT-Mobile: On-Device Large Language Models for User Intent Understanding in Taobao Feed Recommendation
di: Zhang, Bin, et al.
Pubblicazione: (2026)

AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization
di: Jiang, Zhonghua, et al.
Pubblicazione: (2025)

Fast Distributed Inference Serving for Large Language Models
di: Wu, Bingyang, et al.
Pubblicazione: (2023)

MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment
di: Huang, Hanxian, et al.
Pubblicazione: (2026)

Efficient Deployment of Large Language Models on Resource-constrained Devices
di: Yao, Zhiwei, et al.
Pubblicazione: (2025)

RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction
di: Liu, Sihao, et al.
Pubblicazione: (2026)

Large Language Models Inference Engines based on Spiking Neural Networks
di: Balaji, Adarsha, et al.
Pubblicazione: (2025)

ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation
di: Tang, Zihao, et al.
Pubblicazione: (2024)

Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation
di: Wang, Jiawei, et al.
Pubblicazione: (2024)

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
di: Zeng, Chao, et al.
Pubblicazione: (2024)

Efficient Deployment of Vision-Language Models on Mobile Devices: A Case Study on OnePlus 13R
di: Guerrero, Pablo Robin, et al.
Pubblicazione: (2025)

Fast Inference for Augmented Large Language Models
di: Shahout, Rana, et al.
Pubblicazione: (2024)

AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment
di: Fu, Yonggan, et al.
Pubblicazione: (2024)

EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
di: Sanyal, Arnab, et al.
Pubblicazione: (2025)

Scaling On-Device GPU Inference for Large Generative Models
di: Tang, Jiuqiang, et al.
Pubblicazione: (2025)

Fast NF4 Dequantization Kernels for Large Language Model Inference
di: Qi, Xiangbo, et al.
Pubblicazione: (2026)

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference
di: Chen, Yaoqi, et al.
Pubblicazione: (2025)

Resource-Efficient Generative AI Model Deployment in Mobile Edge Networks
di: Liang, Yuxin, et al.
Pubblicazione: (2024)

Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions
di: Niu, Chaoyue, et al.
Pubblicazione: (2025)

Fast and Compact Tsetlin Machine Inference on CPUs Using Instruction-Level Optimization
di: Zeng, Yefan, et al.
Pubblicazione: (2025)

Semantic Trimming and Auxiliary Multi-step Prediction for Generative Recommendation
di: Zhan, Tianyu, et al.
Pubblicazione: (2026)

Graph Neural Networks Automated Design and Deployment on Device-Edge Co-Inference Systems
di: Zhou, Ao, et al.
Pubblicazione: (2024)

Fast-PGM: Fast Probabilistic Graphical Model Learning and Inference
di: Jiang, Jiantong, et al.
Pubblicazione: (2024)

BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
di: Yang, Weiqin, et al.
Pubblicazione: (2026)

A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models
di: Xie, Zuan, et al.
Pubblicazione: (2025)

PLMM: Personal Large Language Models on Mobile Devices
di: Gong, Yuanhao
Pubblicazione: (2023)

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
di: Chitty-Venkata, Krishna Teja, et al.
Pubblicazione: (2024)

Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
di: Xiao, Jie, et al.
Pubblicazione: (2024)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
di: Hu, Xing, et al.
Pubblicazione: (2024)

Deploying Large AI Models on Resource-Limited Devices with Split Federated Learning
di: Qiang, Xianke, et al.
Pubblicazione: (2025)

WebLLM: A High-Performance In-Browser LLM Inference Engine
di: Ruan, Charlie F., et al.
Pubblicazione: (2024)

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
di: Zeng, Chao, et al.
Pubblicazione: (2024)

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices
di: Song, Congzheng, et al.
Pubblicazione: (2025)

PowerInfer-2: Fast Large Language Model Inference on a Smartphone
di: Xue, Zhenliang, et al.
Pubblicazione: (2024)

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
di: Liu, Zechun, et al.
Pubblicazione: (2024)