Cover Image

GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Yanyu, Huang, Ganhong
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Experience Scaling: Post-Deployment Evolution For Large Language Models
by: Yin, Xingkun, et al.
Published: (2025)

Instruction-Driven Game Engines on Large Language Models
by: Wu, Hongqiu, et al.
Published: (2024)

Inference Performance Optimization for Large Language Models on CPUs
by: He, Pujiang, et al.
Published: (2024)

LLM-BI: Towards Fully Automated Bayesian Inference with Large Language Models
by: Huang, Yongchao
Published: (2025)

NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problems
by: Li, Xiaozhe, et al.
Published: (2025)

Enabling Energy-Efficient Deployment of Large Language Models on Memristor Crossbar: A Synergy of Large and Small
by: Wang, Zhehui, et al.
Published: (2024)

A Framework for Rapidly Developing and Deploying Protection Against Large Language Model Attacks
by: Swanda, Adam, et al.
Published: (2025)

Unifying Large Language Models and Knowledge Graphs: A Roadmap
by: Pan, Shirui, et al.
Published: (2023)

Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches
by: Dong, Yanjie, et al.
Published: (2024)

Large Language Models (LLMs) Assisted Wireless Network Deployment in Urban Settings
by: Sevim, Nurullah, et al.
Published: (2024)

Environment-Aware Adaptive Pruning with Interleaved Inference Orchestration for Vision-Language-Action Models
by: Huang, Yuting, et al.
Published: (2026)

Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models
by: Baumann, Nicolas, et al.
Published: (2025)

LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment
by: Yang, Ge, et al.
Published: (2024)

Deploying Multi-task Online Server with Large Language Model
by: Qu, Yincen, et al.
Published: (2024)

Multimodal Survival Analysis with Locally Deployable Large Language Models
by: Gögl, Moritz, et al.
Published: (2026)

UniDM: A Unified Framework for Data Manipulation with Large Language Models
by: Qian, Yichen, et al.
Published: (2024)

Evaluating Search Engines and Large Language Models for Answering Health Questions
by: Fernández-Pichel, Marcos, et al.
Published: (2024)

RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines
by: Yu, Pengfei, et al.
Published: (2025)

SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment
by: Song, Yixin, et al.
Published: (2025)

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
by: Dong, Yixin, et al.
Published: (2024)

TVS Sidekick: Challenges and Practical Insights from Deploying Large Language Models in the Enterprise
by: Lobo, Paula Reyero, et al.
Published: (2025)

Deploying Open-Source Large Language Models: A performance Analysis
by: Bendi-Ouis, Yannis, et al.
Published: (2024)

LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments
by: Huang, Jin, et al.
Published: (2025)

HMVI: Unifying Heterogeneous Attributes with Natural Neighbors for Missing Value Inference
by: Luo, Xiaopeng, et al.
Published: (2026)

Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines
by: Gulati, Akshay, et al.
Published: (2026)

Empowering Intelligent Low-altitude Economy with Large AI Model Deployment
by: Lyu, Zhonghao, et al.
Published: (2025)

Can AI Master Construction Management (CM)? Benchmarking State-of-the-Art Large Language Models on CM Certification Exams
by: Xiong, Ruoxin, et al.
Published: (2025)

FlashOmni: A Unified Sparse Attention Engine for Diffusion Transformers
by: Qiao, Liang, et al.
Published: (2025)

Probing Multimodal Large Language Models for Global and Local Semantic Representations
by: Tao, Mingxu, et al.
Published: (2024)

Large Language Models and Causal Inference in Collaboration: A Survey
by: Liu, Xiaoyu, et al.
Published: (2024)

Can Heterogeneous Language Models Be Fused?
by: Chen, Shilian, et al.
Published: (2026)

Membership Inference Attacks on Tokenizers of Large Language Models
by: Tong, Meng, et al.
Published: (2025)

EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
by: Zhou, Weikang, et al.
Published: (2024)

MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents
by: Zhang, Zeyu, et al.
Published: (2025)

Technique Inference Engine: A Recommender Model to Support Cyber Threat Hunting
by: Turner, Matthew J., et al.
Published: (2025)

Conceptual Schema Inference for Tabular Datasets using Large Language Models
by: Wu, Zhenyu, et al.
Published: (2025)

CASCADE: Case-Based Continual Adaptation for Large Language Models During Deployment
by: Guo, Siyuan, et al.
Published: (2026)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
by: Dai, Jincheng, et al.
Published: (2024)

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models
by: Weng, Zixuan, et al.
Published: (2026)

Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments
by: Du, Yipeng, et al.
Published: (2025)