:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Authors:	Ma, Xindian, Liu, Wenyuan, Zhang, Peng, Xu, Nan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	https://arxiv.org/abs/2406.09897
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SEE: Sememe Entanglement Encoding for Transformer-bases Models Compression
by: Zhang, Jing, et al.
Published: (2024)

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

Context-aware Rotary Position Embedding
by: Veisi, Ali, et al.
Published: (2025)

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
by: Dai, Chang, et al.
Published: (2025)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
by: Wang, Zhenghua, et al.
Published: (2025)

2D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Models
by: Li, Jia-Nan, et al.
Published: (2024)

OMEGA: Optimized Multimodal Position Encoding Index Derivation with Global Adaptive Scaling for Vision-Language Models
by: Huang, Ruoxiang, et al.
Published: (2025)

ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
by: Ma, Xindian, et al.
Published: (2026)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
by: Liu, Feilong
Published: (2026)

Round and Round We Go! What makes Rotary Positional Encodings useful?
by: Barbero, Federico, et al.
Published: (2024)

Selective Rotary Position Embedding
by: Movahedi, Sajad, et al.
Published: (2025)

Long-Context Language Modeling with Parallel Context Encoding
by: Yen, Howard, et al.
Published: (2024)

A$^2$ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization
by: He, Junhui, et al.
Published: (2025)

TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
by: Wu, Bingheng, et al.
Published: (2025)

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
by: Wu, Tong, et al.
Published: (2024)

CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
by: Liu, Wenyuan, et al.
Published: (2024)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
by: Zhang, Zhenyu, et al.
Published: (2024)

DoPE: Denoising Rotary Position Embedding
by: Xiong, Jing, et al.
Published: (2025)

MrRoPE: Mixed-radix Rotary Position Embedding
by: Tian, Qingyuan, et al.
Published: (2026)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
by: Chen, Yuhan, et al.
Published: (2024)

C^2ROPE: Causal Continuous Rotary Positional Encoding for 3D Large Multimodal-Models Reasoning
by: Ye, Guanting, et al.
Published: (2026)

The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval
by: Chiang, Ting-Rui, et al.
Published: (2025)

KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding
by: Shi, Luohe, et al.
Published: (2025)

Benchmarking Rotary Position Embeddings for Automatic Speech Recognition
by: Zhang, Shucong, et al.
Published: (2025)

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability
by: Liao, Baohao, et al.
Published: (2024)

LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data
by: Yang, Cehao, et al.
Published: (2025)

Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding
by: Xiao, Zikai, et al.
Published: (2025)

Marathon: A Race Through the Realm of Long Context with Large Language Models
by: Zhang, Lei, et al.
Published: (2023)

Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models
by: Zhao, Runsong, et al.
Published: (2024)

LaMPE: Length-aware Multi-grained Positional Encoding for Adaptive Long-context Scaling Without Training
by: Zhang, Sikui, et al.
Published: (2025)

CHiRPE: A Step Towards Real-World Clinical NLP with Clinician-Oriented Model Explanations
by: Fong, Stephanie, et al.
Published: (2026)

Group Representational Position Encoding
by: Zhang, Yifan, et al.
Published: (2025)

Wavelet-based Positional Representation for Long Context
by: Oka, Yui, et al.
Published: (2025)

Enhancing Long Context Performance in LLMs Through Inner Loop Query Mechanism
by: Tang, Yimin, et al.
Published: (2024)

Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency
by: Fan, Siqi, et al.
Published: (2025)

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding
by: Zhao, Tiancheng, et al.
Published: (2024)

Length-Aware Rotary Position Embedding for Text-Speech Alignment
by: Kim, Hyeongju, et al.
Published: (2025)

Long Context Alignment with Short Instructions and Synthesized Positions
by: Wu, Wenhao, et al.
Published: (2024)

Do 3D Large Language Models Really Understand 3D Spatial Relationships?
by: Ma, Xianzheng, et al.
Published: (2026)

Cross-Axis Transformer with 3D Rotary Positional Embeddings
by: Erickson, Lily
Published: (2023)