Image de couverture de livre

Adaptive Head Budgeting for Efficient Multi-Head Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Faye, Bilal, Mbaye, Abdoulaye, Azzag, Hanane, Lebbah, Mustapha
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Lightweight Modular Parameter-Efficient Tuning for Open-Vocabulary Object Detection
par: Faye, Bilal, et autres
Publié: (2024)

Prototype-Guided Diffusion: Visual Conditioning without External Memory
par: Faye, Bilal, et autres
Publié: (2025)

Supervised Batch Normalization
par: Faye, Bilal, et autres
Publié: (2024)

Value-Free Policy Optimization via Reward Partitioning
par: Faye, Bilal, et autres
Publié: (2025)

OneEncoder: A Lightweight Framework for Progressive Alignment of Modalities
par: Faye, Bilal, et autres
Publié: (2024)

Unsupervised Adaptive Normalization
par: Faye, Bilal, et autres
Publié: (2024)

Lightweight Cross-Modal Representation Learning
par: Faye, Bilal, et autres
Publié: (2024)

Adaptative Context Normalization: A Boost for Deep Learning in Image Processing
par: Faye, Bilal, et autres
Publié: (2024)

Enhancing Neural Network Representations with Prior Knowledge-Based Normalization
par: Faye, Bilal, et autres
Publié: (2024)

Game Theory Meets Statistical Mechanics in Deep Learning Design
par: Bouchaffra, Djamel, et autres
Publié: (2024)

Context Normalization Layer with Applications
par: Faye, Bilal, et autres
Publié: (2023)

Energy-Regularized Spatial Masking: A Novel Approach to Enhancing Robustness and Interpretability in Vision Models
par: Devynck, Tom, et autres
Publié: (2026)

Coalition Free Energy and Adaptive Precision in Multi-Agent Cooperation
par: Bouchaffra, Djamel, et autres
Publié: (2026)

Distributed MCMC inference for Bayesian Non-Parametric Latent Block Model
par: Khoufache, Reda, et autres
Publié: (2024)

MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote Sensing
par: Radouane, Karim, et autres
Publié: (2025)

NeuroGame Transformer: Gibbs-Inspired Attention Driven by Game Theory and Statistical Physics
par: Bouchaffra, Djamel, et autres
Publié: (2026)

A Collective Variational Principle Unifying Bayesian Inference, Game Theory, and Thermodynamics
par: Bouchaffra, Djamel, et autres
Publié: (2026)

Evaluating the Efficacy of Instance Incremental vs. Batch Learning in Delayed Label Environments: An Empirical Study on Tabular Data Streaming for Fraud Detection
par: Amekoe, Kodjo Mawuena, et autres
Publié: (2024)

Manual Verbalizer Enrichment for Few-Shot Text Classification
par: Nguyen, Quang Anh, et autres
Publié: (2024)

Multi-Head Low-Rank Attention
par: Liu, Songtao, et autres
Publié: (2026)

MoH: Multi-Head Attention as Mixture-of-Head Attention
par: Jin, Peng, et autres
Publié: (2024)

Memorization Capacity of Multi-Head Attention in Transformers
par: Mahdavi, Sadegh, et autres
Publié: (2023)

Interleaved Head Attention
par: Duvvuri, Sai Surya, et autres
Publié: (2026)

Efficient LLMs with AMP: Attention Heads and MLP Pruning
par: Mugnaini, Leandro Giusti, et autres
Publié: (2025)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
par: Li, Cheng, et autres
Publié: (2025)

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
par: Chen, Yilong, et autres
Publié: (2024)

CHAI: Clustered Head Attention for Efficient LLM Inference
par: Agarwal, Saurabh, et autres
Publié: (2024)

Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
par: Donhauser, Konstantin, et autres
Publié: (2025)

Boosting House Price Estimations with Multi-Head Gated Attention
par: Sellam, Zakaria Abdellah, et autres
Publié: (2024)

Geometric Analysis of Token Selection in Multi-Head Attention
par: Mudarisov, Timur, et autres
Publié: (2026)

Improving Transformers with Dynamically Composable Multi-Head Attention
par: Xiao, Da, et autres
Publié: (2024)

Superiority of Multi-Head Attention in In-Context Linear Regression
par: Cui, Yingqian, et autres
Publié: (2024)

Benign Overfitting in Single-Head Attention
par: Magen, Roey, et autres
Publié: (2024)

Multi-Head LatentMoE and Head Parallel: Communication-Efficient and Deterministic MoE Parallelism
par: Cui, Chenwei, et autres
Publié: (2026)

Beyond Parallelism: Synergistic Computational Graph Effects in Multi-Head Attention
par: Borde, Haitz Sáez de Ocáriz
Publié: (2025)

A Capacity-Based Rationale for Multi-Head Attention
par: Adler, Micah
Publié: (2025)

Efficient Image Generation with Variadic Attention Heads
par: Walton, Steven, et autres
Publié: (2022)

Multi-Head Spectral-Adaptive Graph Anomaly Detection
par: Cao, Qingyue, et autres
Publié: (2025)

The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms
par: Otsuka, Hikari, et autres
Publié: (2025)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
par: Musat, Tiberiu
Publié: (2024)