:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Authors:	Zhang, Leo, Martens, James
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	https://arxiv.org/abs/2602.05996
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Improved Operator Learning by Orthogonal Attention
by: Xiao, Zipeng, et al.
Published: (2023)

JPmHC Dynamical Isometry via Orthogonal Hyper-Connections
by: Sengupta, Biswa, et al.
Published: (2026)

Self-Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation
by: Heinsen, Franz A., et al.
Published: (2026)

Tree Cross Attention
by: Feng, Leo, et al.
Published: (2023)

Attention as an RNN
by: Feng, Leo, et al.
Published: (2024)

Aggregating Local Saliency Maps for Semi-Global Explainable Image Classification
by: Hinns, James, et al.
Published: (2025)

DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs
by: Jin, Haolin, et al.
Published: (2025)

PLS in the Mirror of Self-Attention
by: Jiangsheng, et al.
Published: (2026)

Differential Gated Self-Attention
by: Lygizou, Elpiniki Maria, et al.
Published: (2025)

AMO: Adaptive Muon Orthogonalization
by: Zhuang, Xinlin, et al.
Published: (2026)

Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
by: He, Junlin, et al.
Published: (2024)

Disentangling shared and private latent factors in multimodal Variational Autoencoders
by: Märtens, Kaspar, et al.
Published: (2024)

Exclusive Self Attention
by: Zhai, Shuangfei
Published: (2026)

Orthogonal Model Merging
by: Yang, Sihan, et al.
Published: (2026)

FISMO: Fisher-Structured Momentum-Orthogonalized Optimizer
by: Xu, Chenrui, et al.
Published: (2026)

Next-Generation Reservoir Computing for Dynamical Inference
by: Cestnik, Rok, et al.
Published: (2025)

OMPQ: Orthogonal Mixed Precision Quantization
by: Ma, Yuexiao, et al.
Published: (2021)

Temporally Multi-Scale Sparse Self-Attention for Physical Activity Data Imputation
by: Wei, Hui, et al.
Published: (2024)

Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers
by: Chen, Hao, et al.
Published: (2026)

MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
by: Chang, Da, et al.
Published: (2026)

GraphXAIN: Narratives to Explain Graph Neural Networks
by: Cedro, Mateusz, et al.
Published: (2024)

Feature Selection Based on Orthogonal Constraints and Polygon Area
by: Zhang, Zhenxing, et al.
Published: (2024)

Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix
by: Hayase, Tomohiro, et al.
Published: (2025)

Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias
by: Town, James, et al.
Published: (2026)

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

Multi-Domain EEG Representation Learning with Orthogonal Mapping and Attention-based Fusion for Cognitive Load Classification
by: Angkan, Prithila, et al.
Published: (2025)

AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Models
by: Zhang, Yuanyun, et al.
Published: (2026)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
by: Lang, Yicheng, et al.
Published: (2026)

HOFT: Householder Orthogonal Fine-tuning
by: Arcas, Alejandro Moreno, et al.
Published: (2025)

Understanding Differential Transformer Unchains Pretrained Self-Attentions
by: Kong, Chaerin, et al.
Published: (2025)

Theory, Analysis, and Best Practices for Sigmoid Self-Attention
by: Ramapuram, Jason, et al.
Published: (2024)

Securing Pathways with Orthogonal Robots
by: Hoorfar, Hamid, et al.
Published: (2023)

CSAI: Conditional Self-Attention Imputation for Healthcare Time-series
by: Qian, Linglong, et al.
Published: (2023)

Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks
by: Zhang, Tian, et al.
Published: (2026)

Deep Orthogonal Hypersphere Compression for Anomaly Detection
by: Zhang, Yunhe, et al.
Published: (2023)

Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation
by: Racioppo, Peter
Published: (2025)

Quaternion Self-Attention with Shared Scores
by: Yamauchi, Shogo, et al.
Published: (2026)

Algebraic Invariants of Lightning Self-Attention
by: Alexandr, Yulia, et al.
Published: (2026)

Modeling Choice via Self-Attention
by: Ko, Joohwan, et al.
Published: (2023)

Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks
by: Liang, Zhangyong, et al.
Published: (2026)