Cover Image

SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Junnan, Liu, Xinyan, Gao, Peifeng, Qi, Zhaobo, Zhang, Beichen, Zhang, Weigang, Chen, Antoni Bert
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SinkTrack: Attention Sink based Context Anchoring for Large Language Models
by: Liu, Xu, et al.
Published: (2026)

TowerDataset: A Heterogeneous Benchmark for Transmission Corridor Segmentation with a Global-Local Fusion Framework
by: Cui, Xu, et al.
Published: (2026)

AIFIND: Artifact-Aware Interpreting Fine-Grained Alignment for Incremental Face Forgery Detection
by: Wang, Hao, et al.
Published: (2026)

STAND: Semantic Anchoring Constraint with Dual-Granularity Disambiguation for Remote Sensing Image Change Captioning
by: Gong, Yanpei, et al.
Published: (2026)

SinkLoRA: Enhanced Efficiency and Chat Capabilities for Long-Context Large Language Models
by: Zhang, Hengyu
Published: (2024)

SAGE: Sink-Aware Grounded Decoding for Multimodal Hallucination Mitigation
by: Shukla, Tripti, et al.
Published: (2026)

To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
by: Luo, Jiayun, et al.
Published: (2025)

DySink: Dynamic Frame Sinks for Autoregressive Long Video Generation
by: Ye, Bo, et al.
Published: (2026)

Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
by: Sok, Jaewon, et al.
Published: (2026)

Sink-Aware Pruning for Diffusion Language Models
by: Myrzakhan, Aidar, et al.
Published: (2026)

CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction
by: Li, Zixuan, et al.
Published: (2025)

When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
by: Choi, Jiho, et al.
Published: (2026)

Efficient Streaming Language Models with Attention Sinks
by: Xiao, Guangxuan, et al.
Published: (2023)

Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink
by: Liu, Guozhi, et al.
Published: (2026)

When Attention Sink Emerges in Language Models: An Empirical View
by: Gu, Xiangming, et al.
Published: (2024)

Attention Sinks in Diffusion Language Models
by: Rulli, Maximo Eduardo, et al.
Published: (2025)

Sink or Swim
by: Larsen, Karen
Published: (2004)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
by: Zhang, Zihou, et al.
Published: (2026)

Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink
by: Wang, Yining, et al.
Published: (2025)

See What You Are Told: Visual Attention Sink in Large Multimodal Models
by: Kang, Seil, et al.
Published: (2025)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
by: Sun, Shangwen, et al.
Published: (2026)

RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
by: Jin, Ruihan, et al.
Published: (2025)

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
by: Binkowski, Jakub, et al.
Published: (2026)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
by: Wu, Hang, et al.
Published: (2025)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
by: Fu, Zizhuo, et al.
Published: (2026)

Steering Multimodal Large Language Models Decoding for Context-Aware Safety
by: Liu, Zheyuan, et al.
Published: (2025)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
by: Qiu, Zihan, et al.
Published: (2025)

On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs
by: Yoo, Suho, et al.
Published: (2026)

Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos
by: Zhou, Yufan, et al.
Published: (2025)

RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory
by: Liu, Jun, et al.
Published: (2025)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
by: Chen, Yihong, et al.
Published: (2026)

Uncertainty-boosted Robust Video Activity Anticipation
by: Qi, Zhaobo, et al.
Published: (2024)

ICL-Router: In-Context Learned Model Representations for LLM Routing
by: Wang, Chenxu, et al.
Published: (2025)

An Intelligent Energy-Efficient Data Routing Scheme for Wireless Sensor Networks Utilizing Mobile Sink
by: Hassan Al-Mahdi, et al.
Published: (2024)

Attention Sinks and Outliers in Attention Residuals
by: Luo, Haozheng, et al.
Published: (2026)

Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router
by: Shao, Chenyang, et al.
Published: (2025)

On the Existence and Behavior of Secondary Attention Sinks
by: Wong, Jeffrey T. H., et al.
Published: (2025)

EDIT: Enhancing Vision Transformers by Mitigating Attention Sink through an Encoder-Decoder Architecture
by: Feng, Wenfeng, et al.
Published: (2025)

VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understanding
by: Lin, Kuanwei, et al.
Published: (2026)