Cover Image

Improving Transformers using Faithful Positional Encoding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Idé, Tsuyoshi, Labaien, Jokin, Chen, Pin-Yu
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding
by: Li, Hongkang, et al.
Published: (2024)

Cross-Process Defect Attribution using Potential Loss Analysis
by: Idé, Tsuyoshi, et al.
Published: (2025)

Black-Box Anomaly Attribution
by: Idé, Tsuyoshi, et al.
Published: (2023)

Graph Transformer with Disease Subgraph Positional Encoding for Improved Comorbidity Prediction
by: Qin, Xihan, et al.
Published: (2025)

On the Geometry of Positional Encodings in Transformers
by: Cirrincione, Giansalvo
Published: (2026)

Improving Position Encoding of Transformers for Multivariate Time Series Classification
by: Foumani, Navid Mohammadi, et al.
Published: (2023)

Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding
by: Wang, Zhe, et al.
Published: (2024)

Comparing Graph Transformers via Positional Encodings
by: Black, Mitchell, et al.
Published: (2024)

Graph Transformers without Positional Encodings
by: Garg, Ayush
Published: (2024)

Wafer Defect Root Cause Analysis with Partial Trajectory Regression
by: Miyaguchi, Kohei, et al.
Published: (2025)

Sequence-Aware Inline Measurement Attribution for Good-Bad Wafer Diagnosis
by: Miyaguchi, Kohei, et al.
Published: (2025)

Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
by: Ma, Qian, et al.
Published: (2025)

Size Transferability of Graph Transformers with Convolutional Positional Encodings
by: Porras-Valenzuela, Javier, et al.
Published: (2026)

Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
by: Choromanski, Krzysztof Marcin, et al.
Published: (2023)

Benchmarking Positional Encodings for GNNs and Graph Transformers
by: Grötschla, Florian, et al.
Published: (2024)

Tab-PET: Graph-Based Positional Encodings for Tabular Transformers
by: Leng, Yunze, et al.
Published: (2025)

Improving Interpretation Faithfulness for Vision Transformers
by: Hu, Lijie, et al.
Published: (2023)

Decentralized Collaborative Learning Framework with External Privacy Leakage Analysis
by: Idé, Tsuyoshi, et al.
Published: (2024)

Understanding and Improving Laplacian Positional Encodings For Temporal GNNs
by: Galron, Yaniv, et al.
Published: (2025)

UWB TDoA Error Correction using Transformers: Patching and Positional Encoding Strategies
by: Coppens, Dieter, et al.
Published: (2025)

Positional Encoding in Transformer-Based Time Series Models: A Survey
by: Irani, Habib, et al.
Published: (2025)

Position Encoding with Random Float Sampling Enhances Length Generalization of Transformers
by: Shimizu, Atsushi, et al.
Published: (2026)

DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification
by: Li, Chenyang, et al.
Published: (2025)

Cardinality-Regularized Hawkes-Granger Model
by: Idé, Tsuyoshi, et al.
Published: (2022)

Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
by: Xu, Kaichen, et al.
Published: (2025)

Learning Laplacian Positional Encodings for Heterophilous Graphs
by: Ito, Michael, et al.
Published: (2025)

From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
by: Runwal, Bharat, et al.
Published: (2024)

Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings
by: Zuo, Chunsheng, et al.
Published: (2024)

DyWPE: Signal-Aware Dynamic Wavelet Positional Encoding for Time Series Transformers
by: Irani, Habib, et al.
Published: (2025)

RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
by: Kim, Byeongchan, et al.
Published: (2026)

Impact of Positional Encoding: Clean and Adversarial Rademacher Complexity for Transformers under In-Context Regression
by: He, Weiyi, et al.
Published: (2025)

Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers
by: Hallam, Mohamed Amine, et al.
Published: (2026)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
by: Yang, Songlin, et al.
Published: (2025)

Targeted Advertising on Social Networks Using Online Variational Tensor Regression
by: Idé, Tsuyoshi, et al.
Published: (2022)

Algebraic Positional Encodings
by: Kogkalidis, Konstantinos, et al.
Published: (2023)

On the Stability of Expressive Positional Encodings for Graphs
by: Huang, Yinan, et al.
Published: (2023)

Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph
by: Irie, Kazuki
Published: (2024)

Excluding the Target Domain Improves Extrapolation: Deconfounded Hierarchical Physics Constraints
by: Okita, Tsuyoshi
Published: (2026)

SeqPE: Transformer with Sequential Position Encoding
by: Li, Huayang, et al.
Published: (2025)

Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics
by: Ramirez, Ibai, et al.
Published: (2025)