Cover Image

A Capacity-Based Rationale for Multi-Head Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Adler, Micah
Format:	Preprint
Published:	2025
Subjects:	Machine Learning I.2.0
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Combinatorial Interpretability of Neural Computation
by: Adler, Micah, et al.
Published: (2025)

Enhancing Multi-Agent Collaboration with Attention-Based Actor-Critic Policies
by: Belinchon, Hugo Garrido-Lestache, et al.
Published: (2025)

Meta Pattern Concern Score: A Novel Evaluation Measure with Human Values for Multi-classifiers
by: Wang, Yanyun, et al.
Published: (2022)

PCA- and SVM-Grad-CAM for Convolutional Neural Networks: Closed-form Jacobian Expression
by: Omae, Yuto
Published: (2025)

Efficiently Scanning and Resampling Spatio-Temporal Tasks with Irregular Observations
by: Ferenczi, Bryce, et al.
Published: (2024)

Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall Capacity
by: Huang, Ningyuan, et al.
Published: (2025)

Deep Learning-Based Intrusion Detection for Automotive Ethernet: Evaluating & Optimizing Fast Inference Techniques for Deployment on Low-Cost Platform
by: Carmo, Pedro R. X., et al.
Published: (2025)

TSFool: Crafting Highly-Imperceptible Adversarial Time Series through Multi-Objective Attack
by: Wang, Yanyun, et al.
Published: (2022)

Equip Pre-ranking with Target Attention by Residual Quantization
by: Li, Yutong, et al.
Published: (2025)

Virtual Nodes based Heterogeneous Graph Convolutional Neural Network for Efficient Long-Range Information Aggregation
by: Yan, Ranhui, et al.
Published: (2025)

Interpretable Clinical Classification with Kolmogorov-Arnold Networks
by: Almodóvar, Alejandro, et al.
Published: (2025)

Empowering Decision Trees via Shape Function Branching
by: Upadhya, Nakul, et al.
Published: (2025)

Prediction Instability in Machine Learning Ensembles
by: Kedziora, Jeremy
Published: (2024)

Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability
by: Edin, Joakim, et al.
Published: (2024)

Informal Safety Guarantees for Simulated Optimizers Through Extrapolation from Partial Simulations
by: Marks, Luke
Published: (2023)

Interpolation, Extrapolation, Hyperpolation: Generalising into new dimensions
by: Ord, Toby
Published: (2024)

CCDSReFormer: Traffic Flow Prediction with a Criss-Crossed Dual-Stream Enhanced Rectified Transformer Model
by: Shao, Zhiqi, et al.
Published: (2024)

Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards
by: Zhu, Yuxuan, et al.
Published: (2026)

Carefully Structured Compression: Efficiently Managing StarCraft II Data
by: Ferenczi, Bryce, et al.
Published: (2024)

Continuous Wavelet Transform and Siamese Network-Based Anomaly Detection in Multi-variate Semiconductor Process Time Series
by: Dey, Bappaditya, et al.
Published: (2025)

Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?
by: Kajitsuka, Tokio, et al.
Published: (2023)

Intelligent Routing for Sparse Demand Forecasting: A Comparative Evaluation of Selection Strategies
by: Zhang, Qiwen
Published: (2025)

Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown
by: Anand, Emile, et al.
Published: (2025)

Learning Can Converge Stably to the Wrong Belief under Latent Reliability
by: Zhang, Zhipeng, et al.
Published: (2026)

Implicit Counterfactual Data Augmentation for Robust Learning
by: Zhou, Xiaoling, et al.
Published: (2023)

Synthetic Tabular Data Validation: A Divergence-Based Approach
by: Apellániz, Patricia A., et al.
Published: (2024)

MultiPruner: Balanced Structure Removal in Foundation Models
by: Muñoz, J. Pablo, et al.
Published: (2025)

Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach
by: Nguyen, Minh-Duong, et al.
Published: (2025)

An Efficient Gradient-Based Inference Attack for Federated Learning
by: Montaña-Fernández, Pablo, et al.
Published: (2025)

On the Invariants of Softmax Attention
by: Lee, Wonsuk
Published: (2026)

Beyond validation loss: Clinically-tailored optimization metrics improve a model's clinical performance
by: Delahunt, Charles B., et al.
Published: (2026)

Graph Analysis Using a GPU-based Parallel Algorithm: Quantum Clustering
by: Wang, Zhe, et al.
Published: (2023)

How Data Quality Affects Machine Learning Models for Credit Risk Assessment
by: Maurino, Andrea
Published: (2025)

Attention Factors for Statistical Arbitrage
by: Epstein, Elliot L., et al.
Published: (2025)

Non-exchangeable Conformal Prediction for Temporal Graph Neural Networks
by: Wang, Tuo, et al.
Published: (2025)

Study Design and Demystification of Physics Informed Neural Networks for Power Flow Simulation
by: Leyli-abadi, Milad, et al.
Published: (2025)

Aspect-Based Sentiment Analysis for Future Tourism Experiences: A BERT-MoE Framework for Persian User Reviews
by: Taskooh, Hamidreza Kazemi, et al.
Published: (2026)

Social Learning through Interactions with Other Agents: A Survey
by: Hillier, Dylan, et al.
Published: (2024)

A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
by: Yan, Peng, et al.
Published: (2023)

A Physics-embedded Deep Learning Framework for Cloth Simulation
by: Zhao, Zhiwei
Published: (2024)