Cover Image

Improved Operator Learning by Orthogonal Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xiao, Zipeng, Hao, Zhongkai, Lin, Bokai, Deng, Zhijie, Su, Hang
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
by: Lin, Bokai, et al.
Published: (2024)

In-context KV-Cache Eviction for LLMs via Attention-Gate
by: Zeng, Zihao, et al.
Published: (2024)

Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators
by: Cheng, Ze, et al.
Published: (2025)

Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations
by: Cheng, Ze, et al.
Published: (2024)

On the Reuse Bias in Off-Policy Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2022)

Helix: Evolutionary Reinforcement Learning for Open-Ended Scientific Problem Solving
by: Su, Chang, et al.
Published: (2026)

Exploratory Diffusion Model for Unsupervised Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2025)

Task Aware Dreamer for Task Generalization in Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2023)

PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2024)

DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
by: Hao, Zhongkai, et al.
Published: (2024)

MOMA: Masked Orthogonal Matrix Alignment for Zero-Additional-Parameter Model Merging
by: Kong, Fanshuang, et al.
Published: (2024)

Orthogonal Self-Attention
by: Zhang, Leo, et al.
Published: (2026)

Preconditioning for Physics-Informed Neural Networks
by: Liu, Songming, et al.
Published: (2024)

Your Diffusion Model is Secretly a Certifiably Robust Classifier
by: Chen, Huanran, et al.
Published: (2024)

WINO: A Weak-Form Physics Informed Neural Operator for Hyperelasticity on Variable Domains
by: Zhu, Bokai, et al.
Published: (2026)

Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
by: Sun, Guanglong, et al.
Published: (2026)

Label Smoothing Improves Gradient Ascent in LLM Unlearning
by: Pang, Zirui, et al.
Published: (2025)

Exploring the Impact of Temperature Scaling in Softmax for Classification and Adversarial Robustness
by: Xuan, Hao, et al.
Published: (2025)

AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training
by: Lv, Qitan, et al.
Published: (2026)

Orthogonal Uplift Learning with Permutation-Invariant Representations for Combinatorial Treatments
by: Su, Xinyan, et al.
Published: (2026)

Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning
by: Bharthulwar, Sid, et al.
Published: (2025)

A Multi-head Attention Fusion Network for Industrial Prognostics under Discrete Operational Conditions
by: Su, Yuqi, et al.
Published: (2026)

Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
by: Lu, Binghang, et al.
Published: (2026)

Improving Sparse Autoencoder with Dynamic Attention
by: Wang, Dongsheng, et al.
Published: (2026)

Approximation Bounds for Recurrent Neural Networks with Application to Regression
by: Jiao, Yuling, et al.
Published: (2024)

GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention
by: Xue, Tianyang, et al.
Published: (2024)

Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?
by: Su, Weijie
Published: (2025)

Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling
by: Wang, Hong, et al.
Published: (2024)

Improving Multimodal Learning Balance and Sufficiency through Data Remixing
by: Ma, Xiaoyu, et al.
Published: (2025)

Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning
by: Zhu, Shijie, et al.
Published: (2025)

Variational Inference, Entropy, and Orthogonality: A Unified Theory of Mixture-of-Experts
by: Su, Ye, et al.
Published: (2026)

AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation
by: Tan, Hengkai, et al.
Published: (2025)

Improving the Sensitivity of Backdoor Detectors via Class Subspace Orthogonalization
by: Yang, Guangmingmei, et al.
Published: (2025)

Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
by: Xiao, Mingqing, et al.
Published: (2024)

LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning
by: Hu, Yanzhe, et al.
Published: (2026)

Transmit Weights, Not Features: Orthogonal-Basis Aided Wireless Point-Cloud Transmission
by: Chang, Junlin, et al.
Published: (2025)

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning
by: Xie, Zhijie, et al.
Published: (2025)

Adversarial Autoencoders in Operator Learning
by: Enyeart, Dustin, et al.
Published: (2024)

Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model
by: Miao, Yibo, et al.
Published: (2023)

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence
by: Xie, Zhijie, et al.
Published: (2022)