Image de couverture de livre

Exploring the Boundaries of On-Device Inference: When Tiny Falls Short, Go Hierarchical

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Behera, Adarsh Prasad, Daubaris, Paulius, Bravo, Iñaki, Gallego, José, Morabito, Roberto, Widmer, Joerg, Champati, Jaya Prakash Varma
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Improved Decision Module Selection for Hierarchical Inference in Resource-Constrained Edge Devices
par: Behera, Adarsh Prasad, et autres
Publié: (2024)

Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
par: Behera, Adarsh Prasad, et autres
Publié: (2025)

Inference Offloading for Cost-Sensitive Binary Classification at the Edge
par: Moothedath, Vishnu Narayanan, et autres
Publié: (2025)

2D-AoI: Age-of-Information of Distributed Sensors for Spatio-Temporal Processes
par: Fidler, Markus, et autres
Publié: (2024)

Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge
par: Moothedath, Vishnu Narayanan, et autres
Publié: (2023)

Low-Regret and Low-Complexity Learning for Hierarchical Inference
par: Chattopadhyay, Sameep, et autres
Publié: (2025)

Error Bounds for the Network Scale-Up Method
par: Díaz-Aranda, Sergio, et autres
Publié: (2024)

Pico-Cloud: Cloud Infrastructure for Tiny Edge Devices
par: Guri, Mordechai
Publié: (2025)

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
par: Zhang, Haolin, et autres
Publié: (2025)

Inference Acceleration for Large Language Models on CPUs
par: PS, Ditto, et autres
Publié: (2024)

Edge-First Language Model Inference: Models, Metrics, and Tradeoffs
par: Jang, SiYoung, et autres
Publié: (2025)

Future-Proofing Mobile Networks: A Digital Twin Approach to Multi-Signal Management
par: Morabito, Roberto, et autres
Publié: (2024)

Sometimes Painful but Certainly Promising: Feasibility and Trade-offs of Language Model Inference at the Edge
par: Abstreiter, Maximilian, et autres
Publié: (2025)

Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices
par: Pfeiffer, Kilian, et autres
Publié: (2024)

Agentic TinyML for Intent-aware Handover in 6G Wireless Networks
par: Saleh, Alaa, et autres
Publié: (2025)

Tiny but Mighty: A Software-Hardware Co-Design Approach for Efficient Multimodal Inference on Battery-Powered Small Devices
par: Li, Yilong, et autres
Publié: (2025)

NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
par: Wang, Irene, et autres
Publié: (2026)

Where Do the Joules Go? Diagnosing Inference Energy Consumption
par: Chung, Jae-Won, et autres
Publié: (2026)

Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networks
par: Wang, Su, et autres
Publié: (2023)

Distributed On-Device LLM Inference With Over-the-Air Computation
par: Zhang, Kai, et autres
Publié: (2025)

Inference Load-Aware Orchestration for Hierarchical Federated Learning
par: Lackinger, Anna, et autres
Publié: (2024)

Enhancing Predictive Maintenance in Mining Mobile Machinery through a TinyML-enabled Hierarchical Inference Network
par: de la Fuente, Raúl, et autres
Publié: (2024)

PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
par: Liu, Kaiwei, et autres
Publié: (2025)

Evaluating Multi-Instance DNN Inferencing on Multiple Accelerators of an Edge Device
par: Tayal, Mumuksh, et autres
Publié: (2025)

SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
par: Chow, Will
Publié: (2025)

PICO: Pipeline Inference Framework for Versatile CNNs on Diverse Mobile Devices
par: Yang, Xiang, et autres
Publié: (2022)

AdaOper: Energy-efficient and Responsive Concurrent DNN Inference on Mobile Devices
par: Lin, Zheng, et autres
Publié: (2024)

LIME:Accelerating Collaborative Lossless LLM Inference on Memory-Constrained Edge Devices
par: Sun, Mingyu, et autres
Publié: (2025)

Minimizing Age of Detection for a Markov Source over a Lossy Channel
par: Garde, Shivang, et autres
Publié: (2025)

Practical Performance Guarantees for Pipelined DNN Inference
par: Archer, Aaron, et autres
Publié: (2023)

Device Scheduling and Assignment in Hierarchical Federated Learning for Internet of Things
par: Zhang, Tinghao, et autres
Publié: (2024)

Token Level Routing Inference System for Edge Devices
par: She, Jianshu, et autres
Publié: (2025)

Smaller, Smarter, Closer: The Edge of Collaborative Generative AI
par: Morabito, Roberto, et autres
Publié: (2025)

Adaptive Device-Edge Collaboration on DNN Inference in AIoT: A Digital Twin-Assisted Approach
par: Hu, Shisheng, et autres
Publié: (2024)

FlexPie: Accelerate Distributed Inference on Edge Devices with Flexible Combinatorial Optimization[Technical Report]
par: Zhang, Runhua, et autres
Publié: (2025)

Adaptive Stream Processing on Edge Devices through Active Inference
par: Sedlak, Boris, et autres
Publié: (2024)

Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference
par: Prabhakar, Rohan Baskar, et autres
Publié: (2024)

Recipes for Pre-training LLMs with MXFP8
par: Mishra, Asit, et autres
Publié: (2025)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
par: Zhao, Xuanlei, et autres
Publié: (2024)

Communication-Efficient Distributed On-Device LLM Inference Over Wireless Networks
par: Zhang, Kai, et autres
Publié: (2025)