Image de couverture de livre

Poly-attention: a general scheme for higher-order self-attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chakrabarti, Sayak, Pitassi, Toniann, Alman, Josh
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers
par: Chakrabarti, Sayak, et autres
Publié: (2026)

Two Heads Are Better than One: Simulating Large Transformers with Small Ones
par: Yu, Hantao, et autres
Publié: (2025)

Improving Predictor Reliability with Selective Recalibration
par: Zollo, Thomas P., et autres
Publié: (2024)

Reorganizing attention-space geometry with expressive attention
par: Gros, Claudius
Publié: (2024)

Tucker Attention: A generalization of approximate attention mechanisms
par: Klein, Timon, et autres
Publié: (2026)

Supervised learning pays attention
par: Craig, Erin, et autres
Publié: (2025)

Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Models
par: Zollo, Thomas P., et autres
Publié: (2023)

An end-to-end attention-based approach for learning on graphs
par: Buterez, David, et autres
Publié: (2024)

Short window attention enables long-term memorization
par: Cabannes, Loïc, et autres
Publié: (2025)

Cross-attentive Cohesive Subgraph Embedding to Mitigate Oversquashing in GNNs
par: Hossain, Tanvir, et autres
Publié: (2026)

QUEST: A robust attention formulation using query-modulated spherical attention
par: Govindarajan, Hariprasath, et autres
Publié: (2026)

Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention
par: Súkeník, Peter, et autres
Publié: (2026)

GQA-μP: The maximal parameterization update for grouped query attention
par: Chickering, Kyle R., et autres
Publié: (2026)

GRC-Net: Gram Residual Co-attention Net for epilepsy prediction
par: You, Bihao, et autres
Publié: (2025)

Static and multivariate-temporal attentive fusion transformer for readmission risk prediction
par: Sun, Zhe, et autres
Publié: (2024)

TransformerFAM: Feedback attention is working memory
par: Hwang, Dongseong, et autres
Publié: (2024)

Optimal message passing for molecular prediction is simple, attentive and spatial
par: Castaneda-Leautaud, Alma C., et autres
Publié: (2025)

Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learning
par: Barnfield, Nicholas, et autres
Publié: (2026)

Three-dimensional attention Transformer for state evaluation in real-time strategy games
par: Ye, Yanqing, et autres
Publié: (2025)

A foundation model with multi-variate parallel attention to generate neuronal activity
par: Carzaniga, Francesco, et autres
Publié: (2025)

Fusion of Multiscale Features Via Centralized Sparse-attention Network for EEG Decoding
par: Cai, Xiangrui, et autres
Publié: (2025)

ACCORD: Autoregressive Constraint-satisfying Generation for COmbinatorial Optimization with Routing and Dynamic attention
par: Abgaryan, Henrik, et autres
Publié: (2025)

S-JEPA: towards seamless cross-dataset transfer through dynamic spatial attention
par: Guetschel, Pierre, et autres
Publié: (2024)

Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
par: Dayag, Elisha, et autres
Publié: (2025)

Sinkhorn doubly stochastic attention rank decay analysis
par: Lapenna, Michela, et autres
Publié: (2026)

Transformer autoencoder with local attention for sparse and irregular time series with application on risk estimation
par: Rodis, Panteleimon
Publié: (2026)

A standard transformer and attention with linear biases for molecular conformer generation
par: Gurev, Viatcheslav, et autres
Publié: (2025)

CNN-TFT explained by SHAP with multi-head attention weights for time series forecasting
par: Stefenon, Stefano F., et autres
Publié: (2025)

Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization
par: Liu, Shixuan, et autres
Publié: (2024)

Differential privacy from axioms
par: Blanc, Guy, et autres
Publié: (2025)

What are you sinking? A geometric approach on attention sink
par: Ruscio, Valeria, et autres
Publié: (2025)

Physics-integrated generative modeling using attentive planar normalizing flow based variational autoencoder
par: Akhtar, Sheikh Waqas
Publié: (2024)

Towards Auto-Modeling of Formal Verification for NextG Protocols: A Multimodal cross- and self-attention Large Language Model Approach
par: Yang, Jingda, et autres
Publié: (2023)

MoXGATE: Modality-aware cross-attention for multi-omic gastrointestinal cancer sub-type classification
par: Dip, Sajib Acharjee, et autres
Publié: (2025)

Linear attention is (maybe) all you need (to understand transformer optimization)
par: Ahn, Kwangjun, et autres
Publié: (2023)

Gate-level boolean evolutionary geometric attention neural networks
par: Shi, Xianshuai, et autres
Publié: (2025)

SkipSNN: Efficiently Classifying Spike Trains with Event-attention
par: Yin, Hang, et autres
Publié: (2024)

DEDUCE: Multi-head attention decoupled contrastive learning to discover cancer subtypes based on multi-omics data
par: Pan, Liangrui, et autres
Publié: (2023)

Is logical analysis performed by transformers taking place in self-attention or in the fully connected part?
par: Shin, Evgeniy, et autres
Publié: (2025)

Critical attention scaling in long-context transformers
par: Chen, Shi, et autres
Publié: (2025)