Imagen de Portada

SLASH the Sink: Sharpening Structural Attention Inside LLMs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Yiming, Lu, Bin, Wang, Xinbing, Zhou, Chenghu, Jin, Meng
Formato:	Preprint
Publicado:	2026
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Compass: Navigating Global Marine Lead Data Integration through Expert-Guided LLM Agent
por: Liu, Yiming, et al.
Publicado: (2026)

Rethinking Efficient Graph Coarsening via a Non-Selfishness Principle
por: Bai, Xu, et al.
Publicado: (2026)

<SOG_k>: One LLM Token for Explicit Graph Structural Understanding
por: Wu, Jingyao, et al.
Publicado: (2026)

Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression
por: Di, Zijun, et al.
Publicado: (2026)

MTSCI: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputation
por: Zhou, Jianping, et al.
Publicado: (2024)

Temporal Generalization Estimation in Evolving Graphs
por: Lu, Bin, et al.
Publicado: (2024)

MagiNet: Mask-Aware Graph Imputation Network for Incomplete Traffic Data
por: Zhou, Jianping, et al.
Publicado: (2024)

Inductive Reasoning for Temporal Knowledge Graphs with Emerging Entities
por: Zhao, Ze, et al.
Publicado: (2026)

AceParse: A Comprehensive Dataset with Diverse Structured Texts for Academic Literature Parsing
por: Ji, Huawei, et al.
Publicado: (2024)

Attention Sinks and Outliers in Attention Residuals
por: Luo, Haozheng, et al.
Publicado: (2026)

Characterizing the Influence of Topology on Graph Learning Tasks
por: Wu, Kailong, et al.
Publicado: (2024)

Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
por: Queipo-de-Llano, Enrique, et al.
Publicado: (2025)

OXYGENERATOR: Reconstructing Global Ocean Deoxygenation Over a Century with Deep Learning
por: Lu, Bin, et al.
Publicado: (2024)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
por: Sun, Shangwen, et al.
Publicado: (2026)

HuRef: HUman-REadable Fingerprint for Large Language Models
por: Zeng, Boyi, et al.
Publicado: (2023)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
por: Chen, Yihong, et al.
Publicado: (2026)

The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation
por: Carlsson, Fredrik, et al.
Publicado: (2024)

Attention Sinks in Diffusion Language Models
por: Rulli, Maximo Eduardo, et al.
Publicado: (2025)

The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity
por: Li, Siquan, et al.
Publicado: (2026)

Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink
por: Liu, Guozhi, et al.
Publicado: (2026)

Good Idea or Not, Representation of LLM Could Tell
por: Xu, Yi, et al.
Publicado: (2024)

Efficient Streaming Language Models with Attention Sinks
por: Xiao, Guangxuan, et al.
Publicado: (2023)

Spectral Filters, Dark Signals, and Attention Sinks
por: Cancedda, Nicola
Publicado: (2024)

On the Existence and Behavior of Secondary Attention Sinks
por: Wong, Jeffrey T. H., et al.
Publicado: (2025)

When Attention Sink Emerges in Language Models: An Empirical View
por: Gu, Xiangming, et al.
Publicado: (2024)

AutoFAIR : Automatic Data FAIRification via Machine Reading
por: Ma, Tingyan, et al.
Publicado: (2024)

CHAINSFORMER: Numerical Reasoning on Knowledge Graphs from a Chain Perspective
por: Zhao, Ze, et al.
Publicado: (2025)

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
por: Ji, Xiaotong, et al.
Publicado: (2026)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
por: Zhang, Stephen, et al.
Publicado: (2025)

STRUX: An LLM for Decision-Making with Structured Explanations
por: Lu, Yiming, et al.
Publicado: (2024)

Beyond Distribution Sharpening: The Importance of Task Rewards
por: Mittal, Sarthak, et al.
Publicado: (2026)

SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models
por: Liu, Junnan, et al.
Publicado: (2026)

EDIT: Enhancing Vision Transformers by Mitigating Attention Sink through an Encoder-Decoder Architecture
por: Feng, Wenfeng, et al.
Publicado: (2025)

Structured Attention Matters to Multimodal LLMs in Document Understanding
por: Liu, Chang, et al.
Publicado: (2025)

See What You Are Told: Visual Attention Sink in Large Multimodal Models
por: Kang, Seil, et al.
Publicado: (2025)

Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing
por: Ye, Nanyang, et al.
Publicado: (2024)

Shape-Guided Diffusion with Inside-Outside Attention
por: Park, Dong Huk, et al.
Publicado: (2022)

When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures
por: Xu, Yongzhong
Publicado: (2026)

Knowledge Capsules: Structured Nonparametric Memory Units for LLMs
por: Ju, Bin, et al.
Publicado: (2026)

On the Interplay Between Stepsize Tuning and Progressive Sharpening
por: Roulet, Vincent, et al.
Publicado: (2023)