Cover Image

ASAP: Attention Sink Anchored Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Jaehyuk, Kim, Hanyoung, Kim, Yanggee, Lee, Donghun
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Broadband Ground Motion Synthesis by Diffusion Model with Minimal Condition
by: Jung, Jaeheun, et al.
Published: (2024)

IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning
by: Jung, Jaeheun, et al.
Published: (2025)

Catalyst: a Novel Regularizer for Structured Pruning with Auxiliary Extension of Parameter Space
by: Jung, Jaeheun, et al.
Published: (2025)

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference
by: Pathak, Surendra, et al.
Published: (2026)

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
by: Son, Seungwoo, et al.
Published: (2024)

Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs
by: Kim, Kibum, et al.
Published: (2026)

Trust Region Q Adjoint Matching
by: Dong, Yonghoon, et al.
Published: (2026)

ABC3: Active Bayesian Causal Inference with Cohn Criteria in Randomized Experiments
by: Cha, Taehun, et al.
Published: (2024)

ASAP: Amortized Doubly-Stochastic Attention via Sliced Dual Projection
by: Tran, Huy, et al.
Published: (2026)

$t^3$-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergence
by: Kim, Juno, et al.
Published: (2023)

ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate
by: Park, Heewon, et al.
Published: (2025)

Attention Sinks and Outliers in Attention Residuals
by: Luo, Haozheng, et al.
Published: (2026)

Sink-Aware Pruning for Diffusion Language Models
by: Myrzakhan, Aidar, et al.
Published: (2026)

Data-Driven Dimensional Synthesis of Diverse Planar Four-bar Function Generation Mechanisms via Direct Parameterization
by: Kim, Woon Ryong, et al.
Published: (2025)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
by: Chen, Yihong, et al.
Published: (2026)

DiaTool-DPO: Multi-Turn Direct Preference Optimization for Tool-Augmented Large Language Models
by: Jung, Sunghee, et al.
Published: (2025)

Grouped Differential Attention
by: Lim, Junghwan, et al.
Published: (2025)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
by: Fu, Zizhuo, et al.
Published: (2026)

Stochastic Parroting in Temporal Attention -- Regulating the Diagonal Sink
by: Hankemeier, Victoria, et al.
Published: (2026)

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
by: Zuhri, Zayd M. K., et al.
Published: (2025)

Multi-View Node Pruning for Accurate Graph Representation
by: Kim, Hanjin, et al.
Published: (2025)

Are Self-Attentions Effective for Time Series Forecasting?
by: Kim, Dongbin, et al.
Published: (2024)

Structure-Aware Set Transformers: Temporal and Variable-Type Attention Biases for Asynchronous Clinical Time Series
by: Lee, Joohyung, et al.
Published: (2026)

A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention
by: Lee, Heejun, et al.
Published: (2024)

Locality-Aware Redundancy Pruning for LLM Depth Compression
by: Yun, Vincent-Daniel, et al.
Published: (2026)

Learning Where It Matters: Geometric Anchoring for Robust Preference Alignment
by: Cho, Youngjae, et al.
Published: (2026)

SAP: Syntactic Attention Pruning for Transformer-based Language Models
by: Lee, Tzu-Yun, et al.
Published: (2025)

Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
by: Su, Zunhai, et al.
Published: (2026)

On the Existence and Behavior of Secondary Attention Sinks
by: Wong, Jeffrey T. H., et al.
Published: (2025)

The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks
by: Cha, Taehun, et al.
Published: (2026)

OPC: One-Point-Contraction Unlearning Toward Deep Feature Forgetting
by: Jung, Jaeheun, et al.
Published: (2025)

Training-Free Restoration of Pruned Neural Networks
by: Lee, Keonho, et al.
Published: (2025)

Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning
by: Lee, Sungyoung, et al.
Published: (2026)

AdaRank: Adaptive Rank Pruning for Enhanced Model Merging
by: Lee, Chanhyuk, et al.
Published: (2025)

DoMIX: An Efficient Framework for Exploiting Domain Knowledge in Fine-Tuning
by: Kim, Dohoon, et al.
Published: (2025)

SEA: Sparse Linear Attention with Estimated Attention Mask
by: Lee, Heejun, et al.
Published: (2023)

Attention-Based Neural-Augmented Kalman Filter for Legged Robot State Estimation
by: Lee, Seokju, et al.
Published: (2026)

TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation
by: Kim, Junhan, et al.
Published: (2026)

BoA: Attention-aware Post-training Quantization without Backpropagation
by: Kim, Junhan, et al.
Published: (2024)

BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
by: Lee, Gihun, et al.
Published: (2024)