Image de couverture de livre

RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Du, Yufeng, Harris, Phillip, Tian, Minyang, Huerta, Eliu A, Ronanki, Srikanth, Rongali, Subendhu, Galstyan, Aram, Peng, Hao
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
par: Du, Yufeng, et autres
Publié: (2025)

When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
par: Wang, Haonan, et autres
Publié: (2024)

Periodic RoPE for Infinite Context LLMs
par: Huo, Simin
Publié: (2026)

Base of RoPE Bounds Context Length
par: Men, Xin, et autres
Publié: (2024)

Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
par: Li, Zichong, et autres
Publié: (2026)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
par: Zhong, Meizhi, et autres
Publié: (2024)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
par: Liu, Feilong
Publié: (2026)

Q-ROAR: Outlier-Aware Rescaling for RoPE Position Interpolation in Quantized Long-Context LLMs
par: Qiao, Ye, et autres
Publié: (2025)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
par: Li, Haoran, et autres
Publié: (2026)

RoPE-LIME: RoPE-Space Locality + Sparse-K Sampling for Efficient LLM Attribution
par: Picov, Isaac, et autres
Publié: (2026)

Demystifying the Slash Pattern in Attention: The Role of RoPE
par: Cheng, Yuan, et autres
Publié: (2026)

ReRoPE: Repurposing RoPE for Relative Camera Control
par: Li, Chunyang, et autres
Publié: (2026)

Frayed RoPE and Long Inputs: A Geometric Perspective
par: Wertheimer, Davis, et autres
Publié: (2026)

Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
par: Song, Woomin, et autres
Publié: (2025)

Scaling Laws of RoPE-based Extrapolation
par: Liu, Xiaoran, et autres
Publié: (2023)

Rethinking RoPE: A Mathematical Blueprint for N-dimensional Positional Embedding
par: Liu, Haiping, et autres
Publié: (2025)

Resonance RoPE: Improving Context Length Generalization of Large Language Models
par: Wang, Suyuchen, et autres
Publié: (2024)

RAP: KV-Cache Compression via RoPE-Aligned Pruning
par: Xin, Jihao, et autres
Publié: (2026)

LinearARD: Linear-Memory Attention Distillation for RoPE Restoration
par: Yang, Ning, et autres
Publié: (2026)

Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane
par: Liu, Haoyu, et autres
Publié: (2026)

RoPE Attention Can Be Trained in Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
par: Chen, Bo, et autres
Publié: (2024)

Diffusion MRI Transformer with a Diffusion Space Rotary Positional Embedding (D-RoPE)
par: Kung, Gustavo Chau Loo, et autres
Publié: (2026)

Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization
par: Urrutia, Felipe, et autres
Publié: (2026)

Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks
par: Zhang, Yaobo
Publié: (2026)

Position: Spectral GNNs Are Neither Spectral Nor Superior for Node Classification
par: Jiang, Qin, et autres
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

PhaseWrap-RoPE: Bounded Phase-Wrap Positional Scoring Evidence and Hardware-Validation Tooling
par: Fredriksen, Daniel Eric
Publié: (2026)

Adaptive 3D-RoPE: Physics-Aligned Rotary Positional Encoding for Wireless Foundation Models
par: Zhang, Chenyu, et autres
Publié: (2026)

Untwisting RoPE: Frequency Control for Shared Attention in DiTs
par: Mikaeili, Aryan, et autres
Publié: (2026)

A Circular Argument : Does RoPE need to be Equivariant for Vision?
par: van de Geijn, Chase, et autres
Publié: (2025)

On the token distance modeling ability of higher RoPE attention dimension
par: Hong, Xiangyu, et autres
Publié: (2024)

Rotate Both Ways: Time-and-Order RoPE for Generative Recommendation
par: Wei, Xiaokai, et autres
Publié: (2025)

Wainwright and Kwan: Neither Relativism Nor Exclusivism
par: Anthony O'Hear
Publié: (2025)

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models
par: Li, Bozhou, et autres
Publié: (2025)