Copertina

DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gai, Jiading, Zhang, Shuai, Song, Xiang, Wang, Bernie, Karypis, George
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
di: He, Yongjun, et al.
Pubblicazione: (2025)

Extending Input Contexts of Language Models through Training on Segmented Sequences
di: Karypis, Petros, et al.
Pubblicazione: (2023)

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation
di: Fang, Haoyang, et al.
Pubblicazione: (2026)

QuRL: Efficient Reinforcement Learning with Quantized Rollout
di: Li, Yuhang, et al.
Pubblicazione: (2026)

Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
di: Zhu, Qi, et al.
Pubblicazione: (2024)

LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning
di: Zhang, Haoyue, et al.
Pubblicazione: (2025)

RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
di: Gao, Wei, et al.
Pubblicazione: (2025)

Beyond KV Caching: Shared Attention for Efficient LLMs
di: Liao, Bingli, et al.
Pubblicazione: (2024)

XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
di: Vankov, Daniil, et al.
Pubblicazione: (2026)

MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
di: Rhee, Myunghyun, et al.
Pubblicazione: (2025)

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
di: Wang, Luning, et al.
Pubblicazione: (2024)

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning
di: Mavromatis, Costas, et al.
Pubblicazione: (2024)

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
di: Gu, Hao, et al.
Pubblicazione: (2026)

FlashMask: Efficient and Rich Mask Extension of FlashAttention
di: Wang, Guoxia, et al.
Pubblicazione: (2024)

Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection Layers
di: Wang, Yingheng, et al.
Pubblicazione: (2024)

InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
di: Teng, Xin, et al.
Pubblicazione: (2026)

Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
di: Iso, Hayate, et al.
Pubblicazione: (2026)

FlashDP: Private Training Large Language Models with Efficient DP-SGD
di: Wang, Liangyu, et al.
Pubblicazione: (2025)

Heddle: A Distributed Orchestration System for Agentic RL Rollout
di: Zhang, Zili, et al.
Pubblicazione: (2026)

Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharing
di: Amico, Jeffrey, et al.
Pubblicazione: (2025)

Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
di: Xi, Haocheng, et al.
Pubblicazione: (2026)

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
di: Li, Kunxi, et al.
Pubblicazione: (2025)

KV Admission: Learning What to Write for Efficient Long-Context Inference
di: Huang, Yen-Chieh, et al.
Pubblicazione: (2025)

Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model
di: Wu, Jiahao, et al.
Pubblicazione: (2026)

Stable and Efficient Single-Rollout RL for Multimodal Reasoning
di: Liu, Rui, et al.
Pubblicazione: (2025)

MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
di: Li, Yu, et al.
Pubblicazione: (2026)

Train Less, Learn More: Adaptive Efficient Rollout Optimization for Group-Based Reinforcement Learning
di: Zhang, Zhi, et al.
Pubblicazione: (2026)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
di: Wang, Guangtao, et al.
Pubblicazione: (2025)

BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
di: Zhang, Xuechen, et al.
Pubblicazione: (2025)

EchoRL: Reinforcement Learning via Rollout Echoing
di: Bi, Jinhe, et al.
Pubblicazione: (2026)

KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference
di: Gokhale, Sai, et al.
Pubblicazione: (2025)

Efficient Table Retrieval and Understanding with Multimodal Large Language Models
di: Xu, Zhuoyan, et al.
Pubblicazione: (2026)

Understanding Silent Data Corruption in LLM Training
di: Ma, Jeffrey, et al.
Pubblicazione: (2025)

How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
di: Zhu, Rui, et al.
Pubblicazione: (2026)

Enhancing Training Efficiency Using Packing with Flash Attention
di: Kundu, Achintya, et al.
Pubblicazione: (2024)

Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL
di: Zhai, Zhiyuan, et al.
Pubblicazione: (2026)

LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences
di: Wu, Wenbo, et al.
Pubblicazione: (2025)

P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
di: Hui, Mude, et al.
Pubblicazione: (2026)

Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learning
di: Jiang, Zhida, et al.
Pubblicazione: (2026)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
di: Shi, Dachuan, et al.
Pubblicazione: (2025)