Cover Image

Model-Distributed Inference for Large Language Models at the Edge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Macario, Davide, Seferoglu, Hulya, Koyuncu, Erdem
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Early-Exit meets Model-Distributed Inference at Edge Networks
by: Colocrese, Marco, et al.
Published: (2024)

Priority-Aware Model-Distributed Inference at Edge Networks
by: Li, Teng, et al.
Published: (2024)

Improved Generalization Bounds for Communication Efficient Federated Learning
by: Gholami, Peyman, et al.
Published: (2024)

Privacy-Preserving Hierarchical Model-Distributed Inference
by: Dehkordi, Fatemeh Jafarian, et al.
Published: (2024)

A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip
by: Gholami, Peyman, et al.
Published: (2025)

DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates
by: Gholami, Peyman, et al.
Published: (2023)

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
by: Zhang, Xinyuan, et al.
Published: (2024)

Fast Inference for Augmented Large Language Models
by: Shahout, Rana, et al.
Published: (2024)

PRISM: Distributed Inference for Foundation Models at Edge
by: Qazi, Muhammad Azlan, et al.
Published: (2025)

Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare
by: Acikgoz, Emre Can, et al.
Published: (2024)

MoE$^2$: Optimizing Collaborative Inference for Edge Large Language Models
by: Jin, Lyudong, et al.
Published: (2025)

Structure and Redundancy in Large Language Models: A Spectral Study via Random Matrix Theory
by: Ettori, Davide
Published: (2026)

Distributed Threat Intelligence at the Edge Devices: A Large Language Model-Driven Approach
by: Hasan, Syed Mhamudul, et al.
Published: (2024)

Distributed Interpretability and Control for Large Language Models
by: Desai, Dev Arpan, et al.
Published: (2026)

Exploring Graph Structure Comprehension Ability of Multimodal Large Language Models: Case Studies
by: Zhong, Zhiqiang, et al.
Published: (2024)

Efficient Large Language Model Inference with Neural Block Linearization
by: Erdogan, Mete, et al.
Published: (2025)

Automatic Calibration for Membership Inference Attack on Large Language Models
by: Zade, Saleh Zare, et al.
Published: (2025)

Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities
by: Lin, Zheng, et al.
Published: (2023)

Evidence-based Distributional Alignment for Large Language Models
by: Pham, Viet-Thanh, et al.
Published: (2026)

RLAX: Large-Scale, Distributed Reinforcement Learning for Large Language Models on TPUs
by: Zhou, Runlong, et al.
Published: (2025)

MILE: Model-based Intervention Learning
by: Korkmaz, Yigit, et al.
Published: (2025)

WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
by: Chen, Sihan, et al.
Published: (2025)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
by: Li, Tenghui, et al.
Published: (2025)

Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Models
by: Huang, Jerry, et al.
Published: (2024)

Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledge
by: Plecko, Drago, et al.
Published: (2025)

Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
by: Sung, Mingyu, et al.
Published: (2025)

Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
by: Sui, Yuan, et al.
Published: (2025)

ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
by: Zhu, Zijian, et al.
Published: (2026)

Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
by: Fang, Zhiyuan, et al.
Published: (2025)

AnomalyLLM: Few-shot Anomaly Edge Detection for Dynamic Graphs using Large Language Models
by: Liu, Shuo, et al.
Published: (2024)

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
by: Han, Xueyuan, et al.
Published: (2024)

Efficient Inference Using Large Language Models with Limited Human Data: Fine-Tuning then Rectification
by: Wang, Lei, et al.
Published: (2025)

Deep Sequence Models for Predicting Average Shear Wave Velocity from Strong Motion Records
by: Yilmaz, Baris, et al.
Published: (2025)

Challenges and Research Directions for Large Language Model Inference Hardware
by: Ma, Xiaoyu, et al.
Published: (2026)

An Efficient Inference Framework for Early-exit Large Language Models
by: Miao, Ruijie, et al.
Published: (2024)

Revisiting In-context Learning Inference Circuit in Large Language Models
by: Cho, Hakaze, et al.
Published: (2024)

EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
by: Zhang, Shu-Hao, et al.
Published: (2026)

Scaling On-Device GPU Inference for Large Generative Models
by: Tang, Jiuqiang, et al.
Published: (2025)

IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Models
by: Ling, Yiyang, et al.
Published: (2025)

EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices
by: Yi, Rongjie, et al.
Published: (2023)