Image de couverture de livre

RelayAttention for Efficient Large Language Model Serving with Long System Prompts

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhu, Lei, Wang, Xinjiang, Zhang, Wayne, Lau, Rynson W. H.
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Revisiting the Integration of Convolution and Attention for Vision Backbone
par: Zhu, Lei, et autres
Publié: (2024)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
par: Gao, Bin, et autres
Publié: (2024)

Efficient Prompting Methods for Large Language Models: A Survey
par: Chang, Kaiyan, et autres
Publié: (2024)

Relay Decoding: Concatenating Large Language Models for Machine Translation
par: Fu, Chengpeng, et autres
Publié: (2024)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
par: Hu, Junhao, et autres
Publié: (2024)

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
par: Cheng, Xiaoxue, et autres
Publié: (2024)

AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving
par: Wang, Ying, et autres
Publié: (2025)

$A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
par: Zhou, Yuechi, et autres
Publié: (2025)

P/D-Serve: Serving Disaggregated Large Language Model at Scale
par: Jin, Yibo, et autres
Publié: (2024)

Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
par: Tang, Xinyu, et autres
Publié: (2024)

Dynamic Compressing Prompts for Efficient Inference of Large Language Models
par: Hu, Jinwu, et autres
Publié: (2025)

Serve Programs, Not Prompts
par: Gim, In, et autres
Publié: (2025)

LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
par: Yang, Shang, et autres
Publié: (2025)

LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
par: Hu, Zhiyuan, et autres
Publié: (2024)

Efficient Attention Mechanisms for Large Language Models: A Survey
par: Sun, Yutao, et autres
Publié: (2025)

Alignment for Efficient Tool Calling of Large Language Models
par: Xu, Hongshen, et autres
Publié: (2025)

Robustness of Prompting: Enhancing Robustness of Large Language Models Against Prompting Attacks
par: Mu, Lin, et autres
Publié: (2025)

DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception
par: Luo, Run, et autres
Publié: (2024)

D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models
par: Wan, Zhongwei, et autres
Publié: (2024)

PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models
par: Li, Jinyi, et autres
Publié: (2024)

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models
par: Dong, Zican, et autres
Publié: (2023)

HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference
par: Ai, Xuan, et autres
Publié: (2026)

Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models
par: Li, Chengzhengxu, et autres
Publié: (2024)

FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processing
par: Guo, Shoutao, et autres
Publié: (2025)

From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction
par: Zhu, Mingcheng, et autres
Publié: (2026)

Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering
par: Tang, Xinyu, et autres
Publié: (2025)

Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models
par: Pan, Dayan, et autres
Publié: (2025)

IAPT: Instruction-Aware Prompt Tuning for Large Language Models
par: Zhu, Wei, et autres
Publié: (2024)

Active Prompting with Chain-of-Thought for Large Language Models
par: Diao, Shizhe, et autres
Publié: (2023)

Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields
par: Wang, Haoyuan, et autres
Publié: (2024)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
par: You, Zeng, et autres
Publié: (2025)

Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
par: Cheng, Xiaoxue, et autres
Publié: (2025)

SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models
par: Zhao, Weixiang, et autres
Publié: (2024)

Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models
par: Fan, Yuchun, et autres
Publié: (2025)

RelayGen: Intra-Generation Model Switching for Efficient Reasoning
par: Song, Jiwon, et autres
Publié: (2026)

SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
par: Wang, Bailin, et autres
Publié: (2026)

Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
par: Yan, Siyuan, et autres
Publié: (2025)

Sliding Window Attention Training for Efficient Large Language Models
par: Fu, Zichuan, et autres
Publié: (2025)

PromptExp: Multi-granularity Prompt Explanation of Large Language Models
par: Dong, Ximing, et autres
Publié: (2024)

ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models
par: Chen, Hao, et autres
Publié: (2025)