Image de couverture de livre

Minimalist Softmax Attention Provably Learns Constrained Boolean Functions

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Hu, Jerry Yao-Chieh, Zhang, Xiwen, Su, Maojiang, Song, Zhao, Liu, Han
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Universal Approximation with Softmax Attention
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2025)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

On Flow Matching KL Divergence
par: Su, Maojiang, et autres
Publié: (2025)

A Theoretical Analysis of Discrete Flow Matching Generative Models
par: Su, Maojiang, et autres
Publié: (2025)

Attention Mechanism, Max-Affine Partition, and Universal Approximation
par: Liu, Hude, et autres
Publié: (2025)

Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Fast and Low-Cost Genomic Foundation Models via Outlier Removal
par: Luo, Haozheng, et autres
Publié: (2025)

On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond
par: Cao, Yang, et autres
Publié: (2024)

In-Context Deep Learning via Transformer Models
par: Wu, Weimin, et autres
Publié: (2024)

Discrete Flow Matching Policy Optimization
par: Su, Maojiang, et autres
Publié: (2026)

Provable Failure of Language Models in Learning Majority Boolean Logic via Gradient Descent
par: Chen, Bo, et autres
Publié: (2025)

In-Context Algorithm Emulation in Fixed-Weight Transformers
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2025)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
par: Collins, Liam, et autres
Publié: (2024)

Differentially Private Kernel Density Estimation
par: Liu, Erzhi, et autres
Publié: (2024)

Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models
par: Wu, Dennis, et autres
Publié: (2024)

Transformer Approximations from ReLUs
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2026)

On Differentially Private String Distances
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Provably Efficient Exploration in Inverse Constrained Reinforcement Learning
par: Yue, Bo, et autres
Publié: (2024)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
par: Chou, Yuhong, et autres
Publié: (2024)

Scalable-Softmax Is Superior for Attention
par: Nakanishi, Ken M.
Publié: (2025)

Are Hallucinations Bad Estimations?
par: Liu, Hude, et autres
Publié: (2025)

Provable Differentially Private Computation of the Cross-Attention Mechanism
par: Ke, Yekun, et autres
Publié: (2024)

BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model
par: Xu, Chenwei, et autres
Publié: (2024)

Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regression
par: Zuo, Yifei, et autres
Publié: (2025)

On the Invariants of Softmax Attention
par: Lee, Wonsuk
Publié: (2026)

POLO: Preference-Guided Multi-Turn Reinforcement Learning for Lead Optimization
par: Wang, Ziqing, et autres
Publié: (2025)

Boolean Satisfiability via Imitation Learning
par: Zhang, Zewei, et autres
Publié: (2025)

Characteristic Learning for Provable One Step Generation
par: Ding, Zhao, et autres
Publié: (2024)

FLASH-D: FlashAttention with Hidden Softmax Division
par: Alexandridis, Kosmas, et autres
Publié: (2025)

Forgetting Transformer: Softmax Attention with a Forget Gate
par: Lin, Zhixuan, et autres
Publié: (2025)

Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization
par: Rezazadeh, Navid, et autres
Publié: (2026)

Learning Compact Boolean Networks
par: Wang, Shengpu, et autres
Publié: (2026)

The Path Not Taken: RLVR Provably Learns Off the Principals
par: Zhu, Hanqing, et autres
Publié: (2025)

Nonparametric Modern Hopfield Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
par: Yan, Fanqi, et autres
Publié: (2025)

Bellman Unbiasedness: Toward Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation
par: Cho, Taehyun, et autres
Publié: (2024)

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
par: Sheen, Heejune, et autres
Publié: (2024)