Copertina

Unveiling and Controlling Anomalous Attention Distribution in Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yan, Ruiqing, Du, Xingbo, Deng, Haoyu, Zheng, Linghan, Sun, Qiuzhuang, Hu, Jifang, Shao, Yuhang, Jiang, Penghao, Jiang, Jinrong, Zhao, Lian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

RecurFormer: Not All Transformer Heads Need Self-Attention
di: Yan, Ruiqing, et al.
Pubblicazione: (2024)

UniPhy: Unifying Riemannian-Clifford Geometry and Biorthogonal Dynamics for Planetary-Scale Continuous Weather Modeling
di: Yan, Ruiqing, et al.
Pubblicazione: (2026)

Multi-Modal Video Feature Extraction for Popularity Prediction
di: Liu, Haixu, et al.
Pubblicazione: (2025)

Tighnari: Multi-modal Plant Species Prediction Based on Hierarchical Cross-Attention Using Graph-Based and Vision Backbone-Extracted Features
di: Liu, Haixu, et al.
Pubblicazione: (2025)

The Effect of Attention Head Count on Transformer Approximation
di: Yu, Penghao, et al.
Pubblicazione: (2025)

nnY-Net: Swin-NeXt with Cross-Attention for 3D Medical Images Segmentation
di: Liu, Haixu, et al.
Pubblicazione: (2025)

Optimal Abort Policy for Mission-Critical Systems under Imperfect Condition Monitoring
di: Sun, Qiuzhuang, et al.
Pubblicazione: (2025)

Anomalous shift and optical vorticity in the steady photovoltaic current
di: Zhu, Penghao, et al.
Pubblicazione: (2023)

SARIMAX-Based Power Outage Prediction During Extreme Weather Events
di: Ye, Haoran, et al.
Pubblicazione: (2025)

Anomalous Point-Gap Interactions Unveil the Mirage Bath
di: Sun, Yue, et al.
Pubblicazione: (2025)

IPGPhormer: Interpretable Pathology Graph-Transformer for Survival Analysis
di: Tang, Guo, et al.
Pubblicazione: (2025)

InfoFlow: A Framework for Multi-Layer Transformer Analysis
di: Yu, Penghao, et al.
Pubblicazione: (2026)

Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
di: Liu, Yuxi, et al.
Pubblicazione: (2026)

Attentive Convolution: Unifying the Expressivity of Self-Attention with Convolutional Efficiency
di: Yu, Hao, et al.
Pubblicazione: (2025)

The $L_{p}$-Brunn-Minkowski inequalities for variational functionals with $0\leq p<1$
di: Hu, Jinrong
Pubblicazione: (2024)

Uniqueness of solutions to the isotropic $L_{p}$ Gaussian Minkowski problem
di: Hu, Jinrong
Pubblicazione: (2024)

The generalized Gaussian log-Minkowski problem
di: Hu, Jinrong
Pubblicazione: (2024)

The dual Minkowski problem for positive indices
di: Hu, Jinrong
Pubblicazione: (2025)

RulePlanner: All-in-One Reinforcement Learner for Unifying Design Rules in 3D Floorplanning
di: Zhong, Ruizhe, et al.
Pubblicazione: (2026)

Allocation of Parameters in Transformers
di: Yu, Ruoxi, et al.
Pubblicazione: (2025)

IFViT: Interpretable Fixed-Length Representation for Fingerprint Matching via Vision Transformer
di: Qiu, Yuhang, et al.
Pubblicazione: (2024)

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search
di: Deng, Liwei, et al.
Pubblicazione: (2024)

EatGAN: An Edge-Attention Guided Generative Adversarial Network for Single Image Super-Resolution
di: Rao, Penghao, et al.
Pubblicazione: (2025)

SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data
di: Rao, Penghao, et al.
Pubblicazione: (2025)

Google is all you need: Semi-Supervised Transfer Learning Strategy For Light Multimodal Multi-Task Classification Model
di: Liu, Haixu, et al.
Pubblicazione: (2025)

Data Efficient Any Transformer-to-Mamba Distillation via Attention Bridge
di: Wang, Penghao, et al.
Pubblicazione: (2025)

Characterizing circle graphs with binomial partial Petrial polynomials
di: Feng, Ruiqing, et al.
Pubblicazione: (2025)

Online Planning of Power Flows for Power Systems Against Bushfires Using Spatial Context
di: Xu, Jianyu, et al.
Pubblicazione: (2024)

Unveiling the Strong Interaction origin of Baryon Masses with Lattice QCD
di: Hu, Bolun, et al.
Pubblicazione: (2024)

Task Structure Reverses Layerwise State Encoding in Sequence Models
di: Jiang, Yuhang
Pubblicazione: (2026)

PInVerify: An Offline Embodied Benchmark for Active Instance Verification
di: Jiang, Yuhang
Pubblicazione: (2026)

Detection vs. Execution: Single-Bucket Probes Miss Half the Mamba-2 State Sink
di: Jiang, Yuhang
Pubblicazione: (2026)

From the Brunn-Minkowski inequality to a class of generalized Poincaré-type inequalities for torsional rigidity
di: Fang, Niufa, et al.
Pubblicazione: (2023)

Meta-Black-Box Optimization with Bi-Space Landscape Analysis and Dual-Control Mechanism for SAEA
di: Du, Yukun, et al.
Pubblicazione: (2025)

Anomalous Localization Crossovers From the Competition Between Disorder and Lattice Order in Moiré Lattices
di: Qingying Quan, et al.
Pubblicazione: (2026)

Distributed Channel Estimation and Optimization for 6D Movable Antenna: Unveiling Directional Sparsity
di: Shao, Xiaodan, et al.
Pubblicazione: (2024)

Transforming Cities With Digital Governance: Insights and Innovations From China's Urban Landscape
di: Xinyi Du, et al.
Pubblicazione: (2026)

Dynamic Differential Linear Attention: Enhancing Linear Diffusion Transformer for High-Quality Image Generation
di: Cao, Boyuan, et al.
Pubblicazione: (2026)

End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstruction
di: Zhang, Haoyu, et al.
Pubblicazione: (2026)

Capillary John ellipsoid theorem with applications to capillary curvature problems
di: Hu, Jinrong, et al.
Pubblicazione: (2026)