Copertina

Scaling On-Device GPU Inference for Large Generative Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tang, Jiuqiang, Sarokin, Raman, Ignasheva, Ekaterina, Jensen, Grant, Chen, Lin, Lee, Juhyun, Kulik, Andrei, Grundmann, Matthias
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models
di: Deng, Fei, et al.
Pubblicazione: (2024)

Adaptive Parallel Monte Carlo Tree Search for Efficient Test-time Compute Scaling
di: Kim, Hongbeen, et al.
Pubblicazione: (2026)

DiTTO-LLM: Framework for Discovering Topic-based Technology Opportunities via Large Language Model
di: Kim, Wonyoung, et al.
Pubblicazione: (2025)

A Scalable Multi-GPU Framework for Encrypted Large-Model Inference
di: Jayashankar, Siddharth, et al.
Pubblicazione: (2025)

Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
di: Zhao, Yilong, et al.
Pubblicazione: (2025)

CoVSpec: Efficient Device-Edge Co-Inference for Vision-Language Models via Speculative Decoding
di: Jia, Yuanyuan, et al.
Pubblicazione: (2026)

Enabling Delayed-Full Charging Through Transformer-Based Real-Time-to-Departure Modeling for EV Battery Longevity
di: Lee, Yonggeon, et al.
Pubblicazione: (2025)

Simply Trainable Nearest Neighbour Machine Translation with GPU Inference
di: Amer, Hossam, et al.
Pubblicazione: (2024)

Generating Symbolic World Models via Test-time Scaling of Large Language Models
di: Yu, Zhouliang, et al.
Pubblicazione: (2025)

Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
di: Kamahori, Keisuke, et al.
Pubblicazione: (2024)

The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate
di: Oh, Juhyun, et al.
Pubblicazione: (2024)

Large Language Models as Discounted Bayesian Filters
di: Zhang, Jensen, et al.
Pubblicazione: (2025)

Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
di: Banfic, Nenad, et al.
Pubblicazione: (2026)

Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications
di: Esfahanizadeh, Homa, et al.
Pubblicazione: (2024)

KVNAND: Efficient On-Device Large Language Model Inference Using DRAM-Free In-Flash Computing
di: Deng, Lishuo, et al.
Pubblicazione: (2025)

Merino: Entropy-driven Design for Generative Language Models on IoT Devices
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

Belief-Guided Inference Control for Large Language Model Services via Verifiable Observations
di: Yuan, Wenhao, et al.
Pubblicazione: (2026)

POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models
di: Chen, Yi, et al.
Pubblicazione: (2026)

Rethinking the Outlier Distribution in Large Language Models: An In-depth Study
di: Raman, Rahul, et al.
Pubblicazione: (2025)

Recurrent Diffusion for Large-Scale Parameter Generation
di: Wang, Kai, et al.
Pubblicazione: (2025)

Large-Scale Universal Defect Generation: Foundation Models and Datasets
di: Fan, Yuanting, et al.
Pubblicazione: (2026)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
di: Xiong, Chen, et al.
Pubblicazione: (2025)

Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices
di: Ye, Shengyuan, et al.
Pubblicazione: (2025)

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
di: Han, Xueyuan, et al.
Pubblicazione: (2024)

A Contemporary Overview: Trends and Applications of Large Language Models on Mobile Devices
di: Liu, Lianjun, et al.
Pubblicazione: (2024)

Scaling Laws for State Dynamics in Large Language Models
di: Li, Jacob X, et al.
Pubblicazione: (2025)

Scaling Performance of Large Language Model Pretraining
di: Interrante-Grant, Alexander, et al.
Pubblicazione: (2025)

A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems
di: Wu, Qi, et al.
Pubblicazione: (2026)

Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond
di: Oncescu, Costin-Andrei, et al.
Pubblicazione: (2024)

Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data
di: Afzal, Anum, et al.
Pubblicazione: (2024)

Inference Performance Optimization for Large Language Models on CPUs
di: He, Pujiang, et al.
Pubblicazione: (2024)

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
di: Nguyen-Tang, Thanh, et al.
Pubblicazione: (2024)

On The Statistical Complexity of Offline Decision-Making
di: Nguyen-Tang, Thanh, et al.
Pubblicazione: (2025)

MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation
di: Zhang, Haoxuan, et al.
Pubblicazione: (2026)

Measuring and Mitigating Bias in Code Generated by Large Language Models
di: Chen, Yuxi, et al.
Pubblicazione: (2026)

SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling
di: Wang, Haoran, et al.
Pubblicazione: (2025)

Building Large-Scale English-Romanian Literary Translation Resources with Open Models
di: Nadas, Mihai, et al.
Pubblicazione: (2025)

Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
di: Chen, Yushuo, et al.
Pubblicazione: (2024)

Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
di: Mircea, Andrei, et al.
Pubblicazione: (2025)

Constructive Symbolic Reinforcement Learning via Intuitionistic Logic and Goal-Chaining Inference
di: Patrascu, Andrei T.
Pubblicazione: (2025)