Image de couverture de livre

On the Universality of Transformer Architectures; How Much Attention Is Enough?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Abbasi, Amirreza, Hooshmand, Mohsen
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
par: Zhai, Zhiyuan, et autres
Publié: (2026)

Beyond Pipelines: A Fundamental Study on the Rise of Generative-Retrieval Architectures in Web Research
par: Abbasi, Amirereza, et autres
Publié: (2026)

RSAttAE: An Information-Aware Attention-based Autoencoder Recommender System
par: Taromi, Amirhossein Dadashzadeh, et autres
Publié: (2025)

How Much Data is Enough? Optimization of Data Collection for Artifact Detection in EEG Recordings
par: Wang-Nöth, Lu, et autres
Publié: (2024)

How Much Data Is Enough? Uniform Convergence Bounds for Generative & Vision-Language Models under Low-Dimensional Structure
par: Thompson, Paul M.
Publié: (2025)

How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification
par: Jafari, Alireza, et autres
Publié: (2026)

How Much Data is Enough? The Zeta Law of Discoverability in Biomedical Data, featuring the enigmatic Riemann zeta function
par: Thompson, Paul M.
Publié: (2026)

Transformers are Expressive, But Are They Expressive Enough for Regression?
par: Nath, Swaroop, et autres
Publié: (2024)

How Much Is Too Much? Adaptive, Context-Aware Risk Detection in Naturalistic Driving
par: Kalantari, Amir Hossein, et autres
Publié: (2025)

Linear Attention is Enough in Spatial-Temporal Forecasting
par: Ning, Xinyu
Publié: (2024)

How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers
par: Wang, Xiao
Publié: (2026)

Sparsity Moves Computation: How FFN Architecture Reshapes Attention in Small Transformers
par: Smithline, Gabriel, et autres
Publié: (2026)

LUNA: Linear Universal Neural Attention with Generalization Guarantees
par: Shahbazi, Ashkan, et autres
Publié: (2025)

How Much Can Time-related Features Enhance Time Series Forecasting?
par: Zeng, Chaolv, et autres
Publié: (2024)

Self-Attention as Distributional Projection: A Unified Interpretation of Transformer Architecture
par: Mehta, Nihal
Publié: (2025)

AnchorGT: Efficient and Flexible Attention Architecture for Scalable Graph Transformers
par: Zhu, Wenhao, et autres
Publié: (2024)

How Much is Too Much? Exploring LoRA Rank Trade-offs for Retaining Knowledge and Domain Robustness
par: Rathore, Darshita, et autres
Publié: (2025)

PRCD-MAP: Learning How Much to Trust Imperfect Priors in Causal Discovery
par: Shan, Xihang, et autres
Publié: (2026)

How Much is Unseen Depends Chiefly on Information About the Seen
par: Lee, Seongmin, et autres
Publié: (2024)

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models
par: Abbasi, Reza, et autres
Publié: (2024)

Controlling Gender Bias in Retrieval via a Backpack Architecture
par: Afzali, Amirabbas, et autres
Publié: (2025)

Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classification
par: Zhou, Jiajun, et autres
Publié: (2026)

How Much Training Data is Memorized in Overparameterized Autoencoders? An Inverse Problem Perspective on Memorization Evaluation
par: Abitbul, Koren, et autres
Publié: (2023)

Self-Attention as a Parametric Endofunctor: A Categorical Framework for Transformer Architectures
par: O'Neill, Charles
Publié: (2025)

Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough?
par: Pescador-Barrios, Guiomar, et autres
Publié: (2024)

Synthetic Augmentation in Imbalanced Learning: When It Helps, When It Hurts, and How Much to Add
par: Ma, Zhengchi, et autres
Publié: (2026)

Do Large Language Models Know How Much They Know?
par: Prato, Gabriele, et autres
Publié: (2025)

How Much Can We Forget about Data Contamination?
par: Bordt, Sebastian, et autres
Publié: (2024)

Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions
par: Toure, Mame Diarra, et autres
Publié: (2026)

Local Attention Mechanism: Boosting the Transformer Architecture for Long-Sequence Time Series Forecasting
par: Aguilera-Martos, Ignacio, et autres
Publié: (2024)

Realism in Action: Anomaly-Aware Diagnosis of Brain Tumors from Medical Images Using YOLOv8 and DeiT
par: Hashemi, Seyed Mohammad Hossein, et autres
Publié: (2024)

When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate
par: Forest, Florent, et autres
Publié: (2025)

Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction
par: Dokduea, Warayut, et autres
Publié: (2025)

BrainBits: How Much of the Brain are Generative Reconstruction Methods Using?
par: Mayo, David, et autres
Publié: (2024)

Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache
par: Dehghankar, Mohsen, et autres
Publié: (2026)

Is In-Context Universality Enough? MLPs are Also Universal In-Context
par: Kratsios, Anastasis, et autres
Publié: (2025)

How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
par: Schwethelm, Kristian, et autres
Publié: (2026)

Accounting for Uncertainty in Machine Learning Surrogates: A Gauss-Hermite Quadrature Approach to Reliability Analysis
par: Tootchi, Amirreza, et autres
Publié: (2025)

Attention Transfer Is Not Universally Effective for Vision Transformers
par: Qin, Huaiyuan, et autres
Publié: (2026)

EEG Emotion Classification Using an Enhanced Transformer-CNN-BiLSTM Architecture with Dual Attention Mechanisms
par: Karim, S M Rakib UI, et autres
Publié: (2026)