Image de couverture de livre

Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Tasks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Kai, Su, Zhan, Dong, Peijie, Mo, Fengran, Gao, Jianfei, Zhang, ShaoTing, Chen, Kai
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
par: Ma, Da, et autres
Publié: (2024)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
par: Liu, Di, et autres
Publié: (2024)

Evaluating Zero-Shot Long-Context LLM Compression
par: Wang, Chenyu, et autres
Publié: (2024)

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
par: Zhou, Xin, et autres
Publié: (2025)

Conversational Search: From Fundamentals to Frontiers in the LLM Era
par: Mo, Fengran, et autres
Publié: (2025)

ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
par: Liu, Xiang, et autres
Publié: (2025)

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
par: Huang, Chensen, et autres
Publié: (2024)

Language Modeling Using Tensor Trains
par: Su, Zhan, et autres
Publié: (2024)

Bridging the Gap: From Ad-hoc to Proactive Search in Conversations
par: Meng, Chuan, et autres
Publié: (2025)

Boosting Biomedical Concept Extraction by Rule-Based Data Augmentation
par: Shao, Qiwei, et autres
Publié: (2024)

FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
par: Chen, Zhuokun, et autres
Publié: (2026)

Squeezed Attention: Accelerating Long Context Length LLM Inference
par: Hooper, Coleman, et autres
Publié: (2024)

LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
par: Ling, Zhan, et autres
Publié: (2025)

LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts
par: Gu, Zhuohan, et autres
Publié: (2024)

A User-Centric Multi-Intent Benchmark for Evaluating Large Language Models
par: Wang, Jiayin, et autres
Publié: (2024)

Scaling Long-Horizon LLM Agent via Context-Folding
par: Sun, Weiwei, et autres
Publié: (2025)

AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
par: Song, Dinghong, et autres
Publié: (2025)

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
par: Jin, Hongye, et autres
Publié: (2024)

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
par: Xiao, Guangxuan, et autres
Publié: (2024)

MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning?
par: Yan, Kai, et autres
Publié: (2025)

Bridging the Gap between Different Vocabularies for LLM Ensemble
par: Xu, Yangyifan, et autres
Publié: (2024)

ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting
par: Tian, Yuxing, et autres
Publié: (2026)

Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards
par: Jia, Ruipeng, et autres
Publié: (2025)

Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions
par: Jing, Dong, et autres
Publié: (2025)

LLM-Driven Multi-Turn Task-Oriented Dialogue Synthesis for Realistic Reasoning
par: Zhu, Yu, et autres
Publié: (2026)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
par: Zhu, Qianchao, et autres
Publié: (2024)

Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
par: Liu, Yanming, et autres
Publié: (2024)

Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference
par: Wu, Zimeng, et autres
Publié: (2026)

The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
par: Tang, Zhenheng, et autres
Publié: (2025)

DiSRouter: Distributed Self-Routing for LLM Selections
par: Zheng, Hang, et autres
Publié: (2025)

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons
par: Kiruluta, Andrew, et autres
Publié: (2025)

Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration
par: Begin, James, et autres
Publié: (2025)

InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference
par: Pan, Xiurui, et autres
Publié: (2024)

Bridging Human and LLM Judgments: Understanding and Narrowing the Gap
par: Polo, Felipe Maia, et autres
Publié: (2025)

SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
par: Li, Zheng, et autres
Publié: (2025)

Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search
par: Mo, Fengran, et autres
Publié: (2024)

History-Aware Conversational Dense Retrieval
par: Mo, Fengran, et autres
Publié: (2024)

Bridging the Gap: Converting Read Text to Conversational Dialogue
par: Singla, Parshav, et autres
Publié: (2026)

Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention
par: Mo, Shibing, et autres
Publié: (2025)

NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities
par: Li, Mo, et autres
Publié: (2024)