Cover Image

Transformer Reconstructed with Dynamic Value Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Wang, Xiaowei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
by: Wi, Hyowon, et al.
Published: (2025)

Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers
by: Bu, Rui, et al.
Published: (2025)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
by: Brandon, William, et al.
Published: (2024)

Low-Rank Key Value Attention
by: O'Neill, James, et al.
Published: (2026)

Multistability of Self-Attention Dynamics in Transformers
by: Altafini, Claudio
Published: (2025)

Improving Transformers with Dynamically Composable Multi-Head Attention
by: Xiao, Da, et al.
Published: (2024)

Invisible Backdoor Attack Through Singular Value Decomposition
by: Chen, Wenmin, et al.
Published: (2024)

Key and Value Weights Are Probably All You Need: On the Necessity of the Query, Key, Value weight Triplet in Self-Attention Transformers
by: Karbevski, Marko, et al.
Published: (2025)

Macformer: Transformer with Random Maclaurin Feature Attention
by: Guo, Yuhan, et al.
Published: (2024)

Thermodynamic Isomorphism of Transformers: A Lagrangian Approach to Attention Dynamics
by: Kim, Gunn
Published: (2026)

On the Role of Attention Masks and LayerNorm in Transformers
by: Wu, Xinyi, et al.
Published: (2024)

Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape
by: Kim, Juno, et al.
Published: (2024)

Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis
by: Wang, Hanyang, et al.
Published: (2025)

VSFormer: Value and Shape-Aware Transformer with Prior-Enhanced Self-Attention for Multivariate Time Series Classification
by: Xi, Wenjie, et al.
Published: (2024)

Attention Dispersion in Dynamic Graph Transformers: Diagnosis and a Transferable Fix
by: Zhang, Jinhao, et al.
Published: (2026)

Data Efficient Any Transformer-to-Mamba Distillation via Attention Bridge
by: Wang, Penghao, et al.
Published: (2025)

Holographic Transformers for Complex-Valued Signal Processing: Integrating Phase Interference into Self-Attention
by: Huang, Enhao, et al.
Published: (2025)

Attention-Only Transformers via Unrolled Subspace Denoising
by: Wang, Peng, et al.
Published: (2025)

Interdomain Attention: Beyond Token-Level Key-Value Memory
by: Kiyohara, Naoki, et al.
Published: (2026)

Cottention: Linear Transformers With Cosine Attention
by: Mongaras, Gabriel, et al.
Published: (2024)

Transformers with Sparse Attention for Granger Causality
by: Mahesh, Riya, et al.
Published: (2024)

Preconditioned Attention: Enhancing Efficiency in Transformers
by: Saratchandran, Hemanth
Published: (2026)

Graph External Attention Enhanced Transformer
by: Liang, Jianqing, et al.
Published: (2024)

On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
by: Ro, Yeonju, et al.
Published: (2025)

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
by: Tian, Yuandong, et al.
Published: (2023)

Quantum Complex-Valued Self-Attention Model
by: Chen, Fu, et al.
Published: (2025)

First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training
by: Kim, Gyudong, et al.
Published: (2025)

Exploring the Integration of Key-Value Attention Into Pure and Hybrid Transformers for Semantic Segmentation
by: Hwa, DeShin, et al.
Published: (2025)

Efficient Attention via Pre-Scoring: Prioritizing Informative Keys in Transformers
by: Li, Zhexiang, et al.
Published: (2025)

AnchorGT: Efficient and Flexible Attention Architecture for Scalable Graph Transformers
by: Zhu, Wenhao, et al.
Published: (2024)

On the Suboptimality of GP-UCB under Polynomial Effective Optimism
by: Wang, Wenjia, et al.
Published: (2023)

Spark Transformer: Reactivating Sparsity in FFN and Attention
by: You, Chong, et al.
Published: (2025)

The Effect of Attention Head Count on Transformer Approximation
by: Yu, Penghao, et al.
Published: (2025)

InAttention: Linear Context Scaling for Transformers
by: Eisner, Joseph
Published: (2024)

Statistical Test for Attention Map in Vision Transformer
by: Shiraishi, Tomohiro, et al.
Published: (2024)

Memorization Capacity of Multi-Head Attention in Transformers
by: Mahdavi, Sadegh, et al.
Published: (2023)

Optimised Grouped-Query Attention Mechanism for Transformers
by: Chen, Yuang, et al.
Published: (2024)

Spectral Conditioning of Attention Improves Transformer Performance
by: Saratchandran, Hemanth, et al.
Published: (2026)

Stronger Graph Transformer with Regularized Attention Scores
by: Ku, Eugene
Published: (2023)

QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RL
by: Lei, Xing, et al.
Published: (2026)