Cover Image

LongRoPE2: Near-Lossless LLM Context Window Scaling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shang, Ning, Zhang, Li Lyna, Wang, Siyuan, Zhang, Gaokai, Lopez, Gilsinia, Yang, Fan, Chen, Weizhu, Yang, Mao
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
by: Ding, Yiran, et al.
Published: (2024)

LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts
by: Wang, Siyuan, et al.
Published: (2025)

Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
by: Li, Zichong, et al.
Published: (2026)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
by: Zhong, Meizhi, et al.
Published: (2024)

Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
by: Huang, Xijie, et al.
Published: (2023)

rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset
by: Liu, Yifei, et al.
Published: (2025)

Base of RoPE Bounds Context Length
by: Men, Xin, et al.
Published: (2024)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
by: Zhu, Qianchao, et al.
Published: (2024)

Scaling Laws of RoPE-based Extrapolation
by: Liu, Xiaoran, et al.
Published: (2023)

LinearARD: Linear-Memory Attention Distillation for RoPE Restoration
by: Yang, Ning, et al.
Published: (2026)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
by: Li, Haoran, et al.
Published: (2026)

rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
by: Guan, Xinyu, et al.
Published: (2025)

RoPE-LIME: RoPE-Space Locality + Sparse-K Sampling for Efficient LLM Attribution
by: Picov, Isaac, et al.
Published: (2026)

Periodic RoPE for Infinite Context LLMs
by: Huo, Simin
Published: (2026)

Frayed RoPE and Long Inputs: A Geometric Perspective
by: Wertheimer, Davis, et al.
Published: (2026)

RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
by: Du, Yufeng, et al.
Published: (2026)

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
by: Yang, Penghui, et al.
Published: (2025)

When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
by: Wang, Haonan, et al.
Published: (2024)

Demystifying the Slash Pattern in Attention: The Role of RoPE
by: Cheng, Yuan, et al.
Published: (2026)

Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
by: Qi, Zhenting, et al.
Published: (2024)

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
by: Jin, Hongye, et al.
Published: (2024)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
by: Liu, Feilong
Published: (2026)

Resonance RoPE: Improving Context Length Generalization of Large Language Models
by: Wang, Suyuchen, et al.
Published: (2024)

KV-Distill: Nearly Lossless Learnable Context Compression for LLMs
by: Chari, Vivek, et al.
Published: (2025)

Scaling Long-Horizon LLM Agent via Context-Folding
by: Sun, Weiwei, et al.
Published: (2025)

LoPT: Lossless Parallel Tokenization Acceleration for Long Context Inference of Large Language Model
by: Shao, Wei, et al.
Published: (2025)

rStar2-Agent: Agentic Reasoning Technical Report
by: Shang, Ning, et al.
Published: (2025)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
by: Liu, Di, et al.
Published: (2024)

RoPE Attention Can Be Trained in Almost Linear Time
by: Cao, Yang, et al.
Published: (2024)

Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks
by: Zhang, Yaobo
Published: (2026)

GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
by: Kang, Hao, et al.
Published: (2024)

MrRoPE: Mixed-radix Rotary Position Embedding
by: Tian, Qingyuan, et al.
Published: (2026)

Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents
by: Shi, Yaorui, et al.
Published: (2025)

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
by: Huang, Chensen, et al.
Published: (2024)

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
by: Sun, Hanshi, et al.
Published: (2024)

Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization
by: Liu, Yuanye, et al.
Published: (2025)

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
by: Long, Lingkun, et al.
Published: (2026)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
by: Chen, Bo, et al.
Published: (2024)

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
by: Xiao, Guangxuan, et al.
Published: (2024)

LongVILA: Scaling Long-Context Visual Language Models for Long Videos
by: Chen, Yukang, et al.
Published: (2024)