Cover Image

ReAttention: Training-Free Infinite Context with Finite Attention Scope

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Xiaoran, Li, Ruixiao, Guo, Qipeng, Liu, Zhigeng, Song, Yuerong, Lv, Kai, Yan, Hang, Li, Linlin, Liu, Qun, Qiu, Xipeng
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
by: Liu, Xiaoran, et al.
Published: (2025)

Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
by: Song, Yuerong, et al.
Published: (2025)

LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

Thus Spake Long-Context Large Language Model
by: Liu, Xiaoran, et al.
Published: (2025)

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
by: Xu, Zhe, et al.
Published: (2024)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

LongWanjuan: Towards Systematic Measurement for Long Text Quality
by: Lv, Kai, et al.
Published: (2024)

Prism: Spectral-Aware Block-Sparse Attention
by: Wang, Xinghao, et al.
Published: (2026)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
by: Peng, Runyu, et al.
Published: (2026)

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
by: Lv, Kai, et al.
Published: (2023)

DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting
by: Lv, Kai, et al.
Published: (2025)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

Identifying Semantic Induction Heads to Understand In-Context Learning
by: Ren, Jie, et al.
Published: (2024)

Full Parameter Fine-tuning for Large Language Models with Limited Resources
by: Lv, Kai, et al.
Published: (2023)

Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models
by: Liu, Peiju, et al.
Published: (2026)

Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval
by: Zhang, Yuwei, et al.
Published: (2025)

Lag-Relative Sparse Attention In Long Context Training
by: Liang, Manlai, et al.
Published: (2025)

Scaling Laws of RoPE-based Extrapolation
by: Liu, Xiaoran, et al.
Published: (2023)

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
by: Lin, Bin, et al.
Published: (2024)

Sparser Block-Sparse Attention via Token Permutation
by: Wang, Xinghao, et al.
Published: (2025)

FastMCTS: A Simple Sampling Strategy for Data Synthesis
by: Li, Peiji, et al.
Published: (2025)

Towards Efficient Multi-Scale Deformable Attention on NPU
by: Huang, Chenghuan, et al.
Published: (2025)

Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
by: Ji, Tao, et al.
Published: (2025)

Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models
by: Wang, Hongjie, et al.
Published: (2024)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
by: Qiu, Quantong, et al.
Published: (2026)

UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance
by: Ma, Yichuan, et al.
Published: (2025)

FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation
by: He, Siyang, et al.
Published: (2026)

Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
by: Zhang, Yechen, et al.
Published: (2026)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
by: You, Zeng, et al.
Published: (2025)

MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training
by: Li, Wenxuan, et al.
Published: (2025)

Training-Free Long-Context Scaling of Large Language Models
by: An, Chenxin, et al.
Published: (2024)

Re-Attentional Controllable Video Diffusion Editing
by: Wang, Yuanzhi, et al.
Published: (2024)

Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generation
by: Liu, Shengyuan, et al.
Published: (2024)

Attention Is Not Retention: The Orthogonality Constraint in Infinite-Context Architectures
by: Zahn, Oliver, et al.
Published: (2026)

AdaEraser: Training-Free Object Removal via Adaptive Attention Suppression
by: Liu, Dingming
Published: (2026)

AttentionGS: Towards Initialization-Free 3D Gaussian Splatting via Structural Attention
by: Liu, Ziao, et al.
Published: (2025)

ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention
by: Liu, Keli, et al.
Published: (2025)

Melodia: Training-Free Music Editing Guided by Attention Probing in Diffusion Models
by: Yang, Yi, et al.
Published: (2025)

Synthesizing Consistent Novel Views via 3D Epipolar Attention without Re-Training
by: Ye, Botao, et al.
Published: (2025)