Image de couverture de livre

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Abhyankar, Reyna, He, Zijian, Srivatsa, Vikranth, Zhang, Hao, Zhang, Yiying
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Preble: Efficient Distributed Prompt Scheduling for LLM Serving
par: Srivatsa, Vikranth, et autres
Publié: (2024)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
par: Srivatsa, Vikranth, et autres
Publié: (2026)

SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
par: Miao, Xupeng, et autres
Publié: (2023)

DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
par: Huang, You-Liang, et autres
Publié: (2026)

VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU
par: He, Zijian, et autres
Publié: (2026)

Inference Acceleration for Large Language Models on CPUs
par: PS, Ditto, et autres
Publié: (2024)

LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
par: Park, Gunho, et autres
Publié: (2022)

A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine
par: Li, Anran, et autres
Publié: (2026)

HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
par: Dong, Xianzhe, et autres
Publié: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
par: Qian, Yulei, et autres
Publié: (2024)

Federated Cross-Domain Click-Through Rate Prediction With Large Language Model Augmentation
par: Qin, Jiangcheng, et autres
Publié: (2025)

PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
par: Butler, Branden, et autres
Publié: (2024)

Infer-EDGE: Dynamic DNN Inference Optimization in 'Just-in-time' Edge-AI Implementations
par: Mounesan, Motahare, et autres
Publié: (2025)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
par: Shyam, Vasu, et autres
Publié: (2026)

Energy-Efficient Split Learning for Fine-Tuning Large Language Models in Edge Networks
par: Li, Zuguang, et autres
Publié: (2024)

Pier: Efficient Large Language Model pretraining with Relaxed Global Communication
par: Fan, Shuyuan, et autres
Publié: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
par: Chen, Liangkun, et autres
Publié: (2025)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
par: Hu, Junhao, et autres
Publié: (2024)

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
par: Ma, Chenxiang, et autres
Publié: (2025)

PackInfer: Compute- and I/O-Efficient Attention for Batched LLM Inference
par: Ning, Rui, et autres
Publié: (2026)

Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?
par: Wang, Yatong, et autres
Publié: (2026)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
par: Wu, Tian, et autres
Publié: (2025)

PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
par: Yi, Jinjun, et autres
Publié: (2025)

SLO-Aware Scheduling for Large Language Model Inferences
par: Huang, Jinqi, et autres
Publié: (2025)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
par: Duan, Jiangfei, et autres
Publié: (2024)

SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
par: Lin, Zheng, et autres
Publié: (2024)

AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
par: Lin, Wenxiang, et autres
Publié: (2026)

FedSEA-LLaMA: A Secure, Efficient and Adaptive Federated Splitting Framework for Large Language Models
par: Zhang, Zishuai, et autres
Publié: (2025)

Cascadia: An Efficient Cascade Serving System for Large Language Models
par: Jiang, Youhe, et autres
Publié: (2025)

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
par: Chen, Fahao, et autres
Publié: (2025)

RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
par: Jin, Chao, et autres
Publié: (2024)

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
par: Du, Kuntai, et autres
Publié: (2025)

Characterizing Communication Patterns in Distributed Large Language Model Inference
par: Xu, Lang, et autres
Publié: (2025)

EcoLoRA: Communication-Efficient Federated Fine-Tuning of Large Language Models
par: Liu, Han, et autres
Publié: (2025)

Token Level Routing Inference System for Edge Devices
par: She, Jianshu, et autres
Publié: (2025)

Towards Resiliency in Large Language Model Serving with KevlarFlow
par: Qian, Shangshu, et autres
Publié: (2026)

Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
par: Wang, Siqi, et autres
Publié: (2024)

Network Edge Inference for Large Language Models: Principles, Techniques, and Opportunities
par: Chen, Zhixiong, et autres
Publié: (2026)

Unlocking Full Efficiency of Token Filtering in Large Language Model Training
par: Chai, Di, et autres
Publié: (2025)

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization
par: Che, Tianshi, et autres
Publié: (2023)