Cover Image

ExPe: Exact Positional Encodings for Generative Transformer Models with Extrapolating Capabilities

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Datseris, Aleksis, Vassileva, Sylvia, Koychev, Ivan, Boytcheva, Svetla
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Using LLMs for Multilingual Clinical Entity Linking to ICD-10
by: Vassileva, Sylvia, et al.
Published: (2025)

FMI@SU ToxHabits: Evaluating LLMs Performance on Toxic Habit Extraction in Spanish Clinical Texts
by: Vassileva, Sylvia, et al.
Published: (2026)

Team Fusion@ SU@ BC8 SympTEMIST track: transformer-based approach for symptom recognition and linking
by: Grazhdanski, Georgi, et al.
Published: (2026)

Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
by: Zhao, Liang, et al.
Published: (2023)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
by: Zheng, Chuanyang, et al.
Published: (2024)

Length Generalization of Causal Transformers without Position Encoding
by: Wang, Jie, et al.
Published: (2024)

On the Geometry of Positional Encodings in Transformers
by: Cirrincione, Giansalvo
Published: (2026)

Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding
by: Chen, Zhanpeng, et al.
Published: (2025)

FMI_SU_Yotkova_Kastreva at SemEval-2026 Task 13: Lightweight Detection of LLM-Generated Code via Stylometric Signals
by: Yotkova, Elitsa, et al.
Published: (2026)

Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
by: He, Zhenyu, et al.
Published: (2024)

Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding
by: Hayakawa, Daichi, et al.
Published: (2024)

Extrapolation by Association: Length Generalization Transfer in Transformers
by: Cai, Ziyang, et al.
Published: (2025)

Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation
by: Bianchessi, Arthur S., et al.
Published: (2025)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
by: Chen, Yuhan, et al.
Published: (2024)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
by: Lee, Philip Heejun
Published: (2025)

From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers
by: Duan, Shaoxiong, et al.
Published: (2023)

SeqPE: Transformer with Sequential Position Encoding
by: Li, Huayang, et al.
Published: (2025)

Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings
by: Zuo, Chunsheng, et al.
Published: (2024)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
by: Yang, Songlin, et al.
Published: (2025)

Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers
by: Hallam, Mohamed Amine, et al.
Published: (2026)

Disinformation Capabilities of Large Language Models
by: Vykopal, Ivan, et al.
Published: (2023)

Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph
by: Irie, Kazuki
Published: (2024)

Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias
by: Chowdhury, Borun D
Published: (2026)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
by: Wang, Zhenghua, et al.
Published: (2025)

A Morphology-Based Investigation of Positional Encodings
by: Ghosh, Poulami, et al.
Published: (2024)

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
by: Lu, Yi, et al.
Published: (2025)

Learning Extrapolative Sequence Transformations from Markov Chains
by: Hager, Sophia, et al.
Published: (2025)

Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding
by: Zeris, Athanasios
Published: (2026)

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models
by: Das, Rocktim Jyoti, et al.
Published: (2024)

Extrapolation Merging: Keep Improving With Extrapolation and Merging
by: Lin, Yiguan, et al.
Published: (2025)

CLEX: Continuous Length Extrapolation for Large Language Models
by: Chen, Guanzheng, et al.
Published: (2023)

Group Representational Position Encoding
by: Zhang, Yifan, et al.
Published: (2025)

Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
by: Zeris, Athanasios
Published: (2026)

Post-OCR Text Correction for Bulgarian Historical Documents
by: Beshirov, Angel, et al.
Published: (2024)

Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts
by: Mészáros, Anna, et al.
Published: (2024)

On the Encoding of Gender in Transformer-based ASR Representations
by: Krishnan, Aravind, et al.
Published: (2024)

Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data
by: Li, Haolong, et al.
Published: (2024)

On the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibility
by: Tatariya, Kushal, et al.
Published: (2025)

Model Extrapolation Expedites Alignment
by: Zheng, Chujie, et al.
Published: (2024)

3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
by: Ma, Xindian, et al.
Published: (2024)