Copertina

Reorganizing attention-space geometry with expressive attention

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Gros, Claudius
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Small transformer architectures for task switching
di: Gros, Claudius
Pubblicazione: (2025)

Poly-attention: a general scheme for higher-order self-attention
di: Chakrabarti, Sayak, et al.
Pubblicazione: (2026)

Supervised learning pays attention
di: Craig, Erin, et al.
Pubblicazione: (2025)

An end-to-end attention-based approach for learning on graphs
di: Buterez, David, et al.
Pubblicazione: (2024)

Short window attention enables long-term memorization
di: Cabannes, Loïc, et al.
Pubblicazione: (2025)

Tucker Attention: A generalization of approximate attention mechanisms
di: Klein, Timon, et al.
Pubblicazione: (2026)

Cross-attentive Cohesive Subgraph Embedding to Mitigate Oversquashing in GNNs
di: Hossain, Tanvir, et al.
Pubblicazione: (2026)

QUEST: A robust attention formulation using query-modulated spherical attention
di: Govindarajan, Hariprasath, et al.
Pubblicazione: (2026)

Static and multivariate-temporal attentive fusion transformer for readmission risk prediction
di: Sun, Zhe, et al.
Pubblicazione: (2024)

Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention
di: Súkeník, Peter, et al.
Pubblicazione: (2026)

GQA-μP: The maximal parameterization update for grouped query attention
di: Chickering, Kyle R., et al.
Pubblicazione: (2026)

GRC-Net: Gram Residual Co-attention Net for epilepsy prediction
di: You, Bihao, et al.
Pubblicazione: (2025)

TransformerFAM: Feedback attention is working memory
di: Hwang, Dongseong, et al.
Pubblicazione: (2024)

Optimal message passing for molecular prediction is simple, attentive and spatial
di: Castaneda-Leautaud, Alma C., et al.
Pubblicazione: (2025)

S-JEPA: towards seamless cross-dataset transfer through dynamic spatial attention
di: Guetschel, Pierre, et al.
Pubblicazione: (2024)

Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learning
di: Barnfield, Nicholas, et al.
Pubblicazione: (2026)

Three-dimensional attention Transformer for state evaluation in real-time strategy games
di: Ye, Yanqing, et al.
Pubblicazione: (2025)

A foundation model with multi-variate parallel attention to generate neuronal activity
di: Carzaniga, Francesco, et al.
Pubblicazione: (2025)

Fusion of Multiscale Features Via Centralized Sparse-attention Network for EEG Decoding
di: Cai, Xiangrui, et al.
Pubblicazione: (2025)

ACCORD: Autoregressive Constraint-satisfying Generation for COmbinatorial Optimization with Routing and Dynamic attention
di: Abgaryan, Henrik, et al.
Pubblicazione: (2025)

Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
di: Dayag, Elisha, et al.
Pubblicazione: (2025)

Sinkhorn doubly stochastic attention rank decay analysis
di: Lapenna, Michela, et al.
Pubblicazione: (2026)

A standard transformer and attention with linear biases for molecular conformer generation
di: Gurev, Viatcheslav, et al.
Pubblicazione: (2025)

Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization
di: Liu, Shixuan, et al.
Pubblicazione: (2024)

CNN-TFT explained by SHAP with multi-head attention weights for time series forecasting
di: Stefenon, Stefano F., et al.
Pubblicazione: (2025)

Transformer autoencoder with local attention for sparse and irregular time series with application on risk estimation
di: Rodis, Panteleimon
Pubblicazione: (2026)

What are you sinking? A geometric approach on attention sink
di: Ruscio, Valeria, et al.
Pubblicazione: (2025)

Physics-integrated generative modeling using attentive planar normalizing flow based variational autoencoder
di: Akhtar, Sheikh Waqas
Pubblicazione: (2024)

From generative AI to the brain: five takeaways
di: Gros, Claudius
Pubblicazione: (2025)

MoXGATE: Modality-aware cross-attention for multi-omic gastrointestinal cancer sub-type classification
di: Dip, Sajib Acharjee, et al.
Pubblicazione: (2025)

Linear attention is (maybe) all you need (to understand transformer optimization)
di: Ahn, Kwangjun, et al.
Pubblicazione: (2023)

SkipSNN: Efficiently Classifying Spike Trains with Event-attention
di: Yin, Hang, et al.
Pubblicazione: (2024)

Gate-level boolean evolutionary geometric attention neural networks
di: Shi, Xianshuai, et al.
Pubblicazione: (2025)

DEDUCE: Multi-head attention decoupled contrastive learning to discover cancer subtypes based on multi-omics data
di: Pan, Liangrui, et al.
Pubblicazione: (2023)

Critical attention scaling in long-context transformers
di: Chen, Shi, et al.
Pubblicazione: (2025)

Bitformer: An efficient Transformer with bitwise operation-based attention for Big Data Analytics at low-cost low-precision devices
di: Duan, Gaoxiang, et al.
Pubblicazione: (2023)

Differentiable architecture search with multi-dimensional attention for spiking neural networks
di: Man, Yilei, et al.
Pubblicazione: (2024)

HCAF-DTA: drug-target binding affinity prediction with cross-attention fused hypergraph neural networks
di: Li, Jiannuo, et al.
Pubblicazione: (2025)

Physics-informed GNN for medium-high voltage AC power flow with edge-aware attention and line search correction operator
di: Kim, Changhun, et al.
Pubblicazione: (2025)

Topology Reorganized Graph Contrastive Learning with Mitigating Semantic Drift
di: Zhang, Jiaqiang, et al.
Pubblicazione: (2024)