Cover Image

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Yuhan, Lv, Ang, Luan, Jian, Wang, Bin, Liu, Wei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
by: Dai, Chang, et al.
Published: (2025)

HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models
by: Li, Haoran, et al.
Published: (2025)

More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives
by: Zhang, Xiaoqing, et al.
Published: (2025)

Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
by: Lin, Hongzhan, et al.
Published: (2024)

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
by: Chen, Yuhan, et al.
Published: (2023)

SeqPE: Transformer with Sequential Position Encoding
by: Li, Huayang, et al.
Published: (2025)

Scaling Laws of RoPE-based Extrapolation
by: Liu, Xiaoran, et al.
Published: (2023)

STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization
by: Chen, Yuhan, et al.
Published: (2025)

ReachAgent: Enhancing Mobile Agent via Page Reaching and Operation
by: Wu, Qinzhuo, et al.
Published: (2025)

RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
by: Du, Yufeng, et al.
Published: (2026)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
by: Liu, Feilong
Published: (2026)

Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
by: He, Zhenyu, et al.
Published: (2024)

Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains
by: Zhang, Juntian, et al.
Published: (2025)

Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs
by: Ma, Xin, et al.
Published: (2024)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

Position Debiasing Fine-Tuning for Causal Perception in Long-Term Dialogue
by: Fan, Shixuan, et al.
Published: (2024)

Extrapolation Merging: Keep Improving With Extrapolation and Merging
by: Lin, Yiguan, et al.
Published: (2025)

PoPE: Legendre Orthogonal Polynomials Based Position Encoding for Large Language Models
by: Aggarwal, Arpit
Published: (2024)

Positional Encoding via Token-Aware Phase Attention
by: Wang, Yu, et al.
Published: (2025)

ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
by: Wu, Qinzhuo, et al.
Published: (2024)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
by: Li, Haoran, et al.
Published: (2026)

BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism
by: Wu, Qinzhuo, et al.
Published: (2025)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
by: Wang, Zhenghua, et al.
Published: (2025)

MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding
by: Wu, Qinzhuo, et al.
Published: (2024)

Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning
by: Zhou, Yuxuan, et al.
Published: (2025)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
by: Zheng, Chuanyang, et al.
Published: (2024)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
by: Zhao, Liang, et al.
Published: (2023)

Group Representational Position Encoding
by: Zhang, Yifan, et al.
Published: (2025)

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels
by: Wei, Lingxiao, et al.
Published: (2024)

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
by: Jin, Hongye, et al.
Published: (2024)

MobileIPL: Enhancing Mobile Agents Thinking Process via Iterative Preference Learning
by: Huang, Kun, et al.
Published: (2025)

Periodic RoPE for Infinite Context LLMs
by: Huo, Simin
Published: (2026)

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
by: Lu, Yi, et al.
Published: (2025)

Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models
by: Lv, Ang, et al.
Published: (2024)

3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
by: Ma, Xindian, et al.
Published: (2024)

Preference-Aware Memory Update for Long-Term LLM Agents
by: Sun, Haoran, et al.
Published: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
by: Liu, Dong, et al.
Published: (2025)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
by: Deng, Shihan, et al.
Published: (2024)