VRoPE: Rotary Position Embedding for Video Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912678683869184 |
|---|---|
| author | Liu, Zikang Guo, Longteng Tang, Yepeng Yue, Tongtian Cai, Junxian Ma, Kai Liu, Qingbin Chen, Xi Liu, Jing |
| author_facet | Liu, Zikang Guo, Longteng Tang, Yepeng Yue, Tongtian Cai, Junxian Ma, Kai Liu, Qingbin Chen, Xi Liu, Jing |
| contents | Rotary Position Embedding (RoPE) has shown strong performance in text-based Large Language Models (LLMs), but extending it to video remains a challenge due to the intricate spatiotemporal structure of video frames. Existing adaptations, such as RoPE-3D, attempt to encode spatial and temporal dimensions separately but suffer from two major limitations: positional bias in attention distribution and disruptions in video-text transitions. To overcome these issues, we propose Video Rotary Position Embedding (VRoPE), a novel positional encoding method tailored for Video-LLMs. Specifically, we introduce a more balanced encoding strategy that mitigates attention biases, ensuring a more uniform distribution of spatial focus. Additionally, our approach restructures positional indices to ensure a smooth transition between video and text tokens. Extensive experiments on different models demonstrate that VRoPE consistently outperforms previous RoPE variants, achieving significant improvements in video understanding, temporal reasoning, and retrieval tasks. Code is available at https://github.com/johncaged/VRoPE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11664 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VRoPE: Rotary Position Embedding for Video Large Language Models Liu, Zikang Guo, Longteng Tang, Yepeng Yue, Tongtian Cai, Junxian Ma, Kai Liu, Qingbin Chen, Xi Liu, Jing Artificial Intelligence Rotary Position Embedding (RoPE) has shown strong performance in text-based Large Language Models (LLMs), but extending it to video remains a challenge due to the intricate spatiotemporal structure of video frames. Existing adaptations, such as RoPE-3D, attempt to encode spatial and temporal dimensions separately but suffer from two major limitations: positional bias in attention distribution and disruptions in video-text transitions. To overcome these issues, we propose Video Rotary Position Embedding (VRoPE), a novel positional encoding method tailored for Video-LLMs. Specifically, we introduce a more balanced encoding strategy that mitigates attention biases, ensuring a more uniform distribution of spatial focus. Additionally, our approach restructures positional indices to ensure a smooth transition between video and text tokens. Extensive experiments on different models demonstrate that VRoPE consistently outperforms previous RoPE variants, achieving significant improvements in video understanding, temporal reasoning, and retrieval tasks. Code is available at https://github.com/johncaged/VRoPE. |
| title | VRoPE: Rotary Position Embedding for Video Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2502.11664 |