Cover Image

A Mathematical Explanation of Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tai, Xue-Cheng, Liu, Hao, Li, Lingfeng, Chan, Raymond H.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Mathematical Guide to Operator Learning
by: Boullé, Nicolas, et al.
Published: (2023)

AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures
by: Gao, Yihang, et al.
Published: (2024)

STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem
by: Wang, Hong, et al.
Published: (2025)

Deriving Transformer Architectures as Implicit Multinomial Regression
by: Actor, Jonas A., et al.
Published: (2025)

FAST: Factorizable Attention for Speeding up Transformers
by: Gerami, Armin, et al.
Published: (2024)

Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers
by: Zhou, Hang, et al.
Published: (2024)

Accelerating Matrix Diagonalization through Decision Transformers with Epsilon-Greedy Optimization
by: Bhatta, Kshitij, et al.
Published: (2024)

P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics
by: Wang, Qi, et al.
Published: (2024)

Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
by: Li, Qinglun, et al.
Published: (2025)

Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling
by: Wang, Hong, et al.
Published: (2024)

Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning
by: Mustajab, Abdul Hannan, et al.
Published: (2024)

HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions
by: Zheng, Haoyang, et al.
Published: (2023)

PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws
by: Yang, Liu, et al.
Published: (2024)

DeepContour: A Hybrid Deep Learning Framework for Accelerating Generalized Eigenvalue Problem Solving via Efficient Contour Design
by: Chen, Yeqiu, et al.
Published: (2025)

CATO: Charted Attention for Neural PDE Operators
by: Cheng, Chun-Wun, et al.
Published: (2026)

Branching Strategies Based on Subgraph GNNs: A Study on Theoretical Promise versus Practical Reality
by: Zhou, Junru, et al.
Published: (2025)

Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Study
by: Ke, Yekun, et al.
Published: (2024)

KAN-GCN: Combining Kolmogorov-Arnold Network with Graph Convolution Network for an Accurate Ice Sheet Emulator
by: Liu, Zesheng, et al.
Published: (2025)

Mitigating spectral bias for the multiscale operator learning
by: Liu, Xinliang, et al.
Published: (2022)

Deep Learning-Enhanced Preconditioning for Efficient Conjugate Gradient Solvers in Large-Scale PDE Systems
by: Li, Rui, et al.
Published: (2024)

Learning to Discover Iterative Spectral Algorithms
by: Liu, Zihang, et al.
Published: (2026)

Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
by: Islam, Chashi Mahiul, et al.
Published: (2026)

Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
by: Yan, Mingsong, et al.
Published: (2026)

Neural Operators with Localized Integral and Differential Kernels
by: Liu-Schiaffini, Miguel, et al.
Published: (2024)

ProFlow: Zero-Shot Physics-Consistent Sampling via Proximal Flow Guidance
by: Yu, Zichao, et al.
Published: (2026)

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
by: Jentzen, Arnulf, et al.
Published: (2023)

BWLer: Barycentric Weight Layer Elucidates a Precision-Conditioning Tradeoff for PINNs
by: Liu, Jerry, et al.
Published: (2025)

IFNSO: Iteration-Free Newton-Schulz Orthogonalization
by: Hu, Chen, et al.
Published: (2026)

FlashSinkhorn: IO-Aware Entropic Optimal Transport on GPU
by: Ye, Felix X. -F., et al.
Published: (2026)

A Practical Approach to Causal Inference over Time
by: Cinquini, Martina, et al.
Published: (2024)

A Dimensionality Reduction Approach for Convolutional Neural Networks
by: Meneghetti, Laura, et al.
Published: (2021)

Defining Foundation Models for Computational Science: A Call for Clarity and Rigor
by: Choi, Youngsoo, et al.
Published: (2025)

Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis
by: Cheng, Long, et al.
Published: (2025)

Learning Explicitly Conditioned Sparsifying Transforms
by: Pătraşcu, Andrei, et al.
Published: (2024)

A practical existence theorem for reduced order models based on convolutional autoencoders
by: Franco, Nicola Rares, et al.
Published: (2024)

Learning Semilinear Neural Operators : A Unified Recursive Framework For Prediction And Data Assimilation
by: Singh, Ashutosh, et al.
Published: (2024)

Accelerated Gradient-based Design Optimization Via Differentiable Physics-Informed Neural Operator: A Composites Autoclave Processing Case Study
by: Patel, Janak M., et al.
Published: (2025)

Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning
by: Berner, Julius, et al.
Published: (2025)

Truncated Matrix Completion - An Empirical Study
by: Naik, Rishhabh, et al.
Published: (2025)

Random weights of DNNs and emergence of fixed points
by: Berlyand, L., et al.
Published: (2025)