Image de couverture de livre

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zheng, Wenhao, Chen, Yixiao, Zhang, Weitong, Kundu, Souvik, Li, Yun, Liu, Zhengzhong, Xing, Eric P., Wang, Hongyi, Yao, Huaxiu
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence Machine Learning Performance
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Token Level Routing Inference System for Edge Devices
par: She, Jianshu, et autres
Publié: (2025)

Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
par: Dong, Ximing, et autres
Publié: (2026)

SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
par: Ma, Ming, et autres
Publié: (2025)

Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
par: Moslem, Yasmin, et autres
Publié: (2026)

DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
par: Lee, Younjoo, et autres
Publié: (2026)

Iterative Layer Pruning for Efficient Translation Inference
par: Moslem, Yasmin, et autres
Publié: (2025)

Model Compression and Efficient Inference for Large Language Models: A Survey
par: Wang, Wenxiao, et autres
Publié: (2024)

R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing
par: Fu, Tianyu, et autres
Publié: (2025)

LFED: A Literary Fiction Evaluation Dataset for Large Language Models
par: Yu, Linhao, et autres
Publié: (2024)

Layer Importance and Hallucination Analysis in Large Language Models via Enhanced Activation Variance-Sparsity
par: Song, Zichen, et autres
Publié: (2024)

Green AI: Exploring Carbon Footprints, Mitigation Strategies, and Trade Offs in Large Language Model Training
par: Liu, Vivian, et autres
Publié: (2024)

Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations
par: Tyukin, Georgy
Publié: (2024)

Priority Sampling of Large Language Models for Compilers
par: Grubisic, Dejan, et autres
Publié: (2024)

Harnessing Chain-of-Thought Metadata for Task Routing and Adversarial Prompt Detection
par: Marinelli, Ryan, et autres
Publié: (2025)

It's Not Easy Being Green: On the Energy Efficiency of Programming Languages
par: van Kempen, Nicolas, et autres
Publié: (2024)

Performance Characterization of Expert Router for Scalable LLM Inference
par: Pichlmeier, Josef, et autres
Publié: (2024)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
par: Hu, Junhao, et autres
Publié: (2024)

SuperCoder: Assembly Program Superoptimization with Large Language Models
par: Wei, Anjiang, et autres
Publié: (2025)

GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference
par: Ziller, Thomas, et autres
Publié: (2026)

Accelerating Diffusion LLMs via Adaptive Parallel Decoding
par: Israel, Daniel, et autres
Publié: (2025)

FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
par: Wu, Wenhao, et autres
Publié: (2026)

ModeSwitch-LLM: A Lightweight Phase-Aware Controller for Cross-Mode LLM Inference on a Single GPU
par: Sunesh, Aman, et autres
Publié: (2026)

Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
par: Benazir, Afsara, et autres
Publié: (2025)

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
par: Hendria, Willy Fitra
Publié: (2026)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
par: Shao, Zishan, et autres
Publié: (2025)

AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms
par: Chen, Feiyang, et autres
Publié: (2025)

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache
par: Du, Dayou, et autres
Publié: (2025)

Meta-Metrics and Best Practices for System-Level Inference Performance Benchmarking
par: Salaria, Shweta, et autres
Publié: (2025)

Systematic Evaluation of Optimization Techniques for Long-Context Language Models
par: Ahmed, Ammar, et autres
Publié: (2025)

LOOPerSet: A Large-Scale Dataset for Data-Driven Polyhedral Compiler Optimization
par: Merouani, Massinissa, et autres
Publié: (2025)

Optimizing Layout of Recursive Datatypes with Marmoset
par: Singhal, Vidush, et autres
Publié: (2024)

Evaluating Compiler Optimization Impacts on zkVM Performance
par: Gassmann, Thomas, et autres
Publié: (2025)

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning
par: Singh, Raul, et autres
Publié: (2025)

Insum: Sparse GPU Kernels Simplified and Optimized with Indirect Einsums
par: Won, Jaeyeon, et autres
Publié: (2025)

KG-EDAS: A Meta-Metric Framework for Evaluating Knowledge Graph Completion Models
par: Gul, Haji, et autres
Publié: (2025)

Rule-Based Graph Programs Matching the Time Complexity of Imperative Algorithms
par: Alaoui, Ziad Ismaili, et autres
Publié: (2025)

Bench360: Benchmarking Local LLM Inference from 360 Degrees
par: Stuhlmann, Linus, et autres
Publié: (2025)

SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Version
par: Pham, Nghiem Thanh, et autres
Publié: (2025)

Two-Timescale Dynamic Service Deployment and Task Scheduling with Spatiotemporal Collaboration in Mobile Edge Networks
par: Li, Yang, et autres
Publié: (2025)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
par: Zhang, Hang, et autres
Publié: (2025)