Image de couverture de livre

Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tang, Zecheng, Qiu, Quantong, Yang, Yi, Hong, Zhiyi, Xiang, Haiya, Liu, Kebin, Dang, Qingqing, Li, Juntao, Zhang, Min
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
par: Qiu, Quantong, et autres
Publié: (2026)

LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling
par: Tang, Zecheng, et autres
Publié: (2025)

LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation framework
par: Tang, Zecheng, et autres
Publié: (2025)

Revealing and Mitigating Over-Attention in Knowledge Editing
par: Wang, Pinzheng, et autres
Publié: (2025)

Accurate KV Cache Quantization with Outlier Tokens Tracing
par: Su, Yi, et autres
Publié: (2025)

CaliDrop: KV Cache Compression with Calibration
par: Su, Yi, et autres
Publié: (2025)

Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
par: Lin, Chaofan, et autres
Publié: (2025)

MemLong: Memory-Augmented Retrieval for Long Text Modeling
par: Liu, Weijie, et autres
Publié: (2024)

Revealing and Mitigating the Local Pattern Shortcuts of Mamba
par: You, Wangjie, et autres
Publié: (2024)

LOGO -- Long cOntext aliGnment via efficient preference Optimization
par: Tang, Zecheng, et autres
Publié: (2024)

L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?
par: Tang, Zecheng, et autres
Publié: (2024)

Rethinking Negative Instances for Generative Named Entity Recognition
par: Ding, Yuyang, et autres
Publié: (2024)

Revisiting Long-context Modeling from Context Denoising Perspective
par: Tang, Zecheng, et autres
Publié: (2025)

MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
par: Ji, Baibei, et autres
Publié: (2026)

Crisp Attention: Regularizing Transformers via Structured Sparsity
par: Gandhi, Sagar, et autres
Publié: (2025)

Efficient Sparse Attention needs Adaptive Token Release
par: Zhang, Chaoran, et autres
Publié: (2024)

Accelerating Prefilling via Decoding-time Contribution Sparsity
par: He, Zhiyuan, et autres
Publié: (2025)

STS: Efficient Sparse Attention with Speculative Token Sparsity
par: Xu, Ceyu, et autres
Publié: (2026)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
par: Qiu, Zihan, et autres
Publié: (2025)

CMD: a framework for Context-aware Model self-Detoxification
par: Tang, Zecheng, et autres
Publié: (2023)

Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models
par: Xiang, Yang, et autres
Publié: (2025)

When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoning
par: Xiang, Yang, et autres
Publié: (2026)

MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
par: Zhou, Keyan, et autres
Publié: (2025)

Post-Training Sparse Attention with Double Sparsity
par: Yang, Shuo, et autres
Publié: (2024)

OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning
par: Qiao, Dan, et autres
Publié: (2024)

Sirius: Contextual Sparsity with Correction for Efficient LLMs
par: Zhou, Yang, et autres
Publié: (2024)

ART: Attention Run-time Termination for Efficient Large Language Model Decoding
par: Qiu, Chen, et autres
Publié: (2026)

Attention Condensation via Sparsity Induced Regularized Training
par: Sason, Eli, et autres
Publié: (2025)

HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference
par: Ai, Xuan, et autres
Publié: (2026)

AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
par: Yang, Leixin, et autres
Publié: (2023)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
par: Zhao, Bowen, et autres
Publié: (2024)

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
par: Yuan, Jiayi, et autres
Publié: (2025)

LongFlow: Efficient KV Cache Compression for Reasoning Models
par: Su, Yi, et autres
Publié: (2026)

OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure
par: Wang, Jikai, et autres
Publié: (2024)

Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
par: Tian, Zhenxu, et autres
Publié: (2026)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
par: Tang, Yehui, et autres
Publié: (2025)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
par: Tang, Jiaming, et autres
Publié: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
par: Yang, Songlin, et autres
Publié: (2023)

Parallel Loop Transformer for Efficient Test-Time Computation Scaling
par: Wu, Bohong, et autres
Publié: (2025)

MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models
par: Tang, Zecheng, et autres
Publié: (2026)