Imagen de Portada

Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tang, Zecheng, Qiu, Quantong, Yang, Yi, Hong, Zhiyi, Xiang, Haiya, Liu, Kebin, Dang, Qingqing, Li, Juntao, Zhang, Min
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
por: Qiu, Quantong, et al.
Publicado: (2026)

LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling
por: Tang, Zecheng, et al.
Publicado: (2025)

LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation framework
por: Tang, Zecheng, et al.
Publicado: (2025)

Revealing and Mitigating Over-Attention in Knowledge Editing
por: Wang, Pinzheng, et al.
Publicado: (2025)

Accurate KV Cache Quantization with Outlier Tokens Tracing
por: Su, Yi, et al.
Publicado: (2025)

CaliDrop: KV Cache Compression with Calibration
por: Su, Yi, et al.
Publicado: (2025)

Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
por: Lin, Chaofan, et al.
Publicado: (2025)

MemLong: Memory-Augmented Retrieval for Long Text Modeling
por: Liu, Weijie, et al.
Publicado: (2024)

Revealing and Mitigating the Local Pattern Shortcuts of Mamba
por: You, Wangjie, et al.
Publicado: (2024)

LOGO -- Long cOntext aliGnment via efficient preference Optimization
por: Tang, Zecheng, et al.
Publicado: (2024)

L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?
por: Tang, Zecheng, et al.
Publicado: (2024)

Rethinking Negative Instances for Generative Named Entity Recognition
por: Ding, Yuyang, et al.
Publicado: (2024)

Revisiting Long-context Modeling from Context Denoising Perspective
por: Tang, Zecheng, et al.
Publicado: (2025)

MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
por: Ji, Baibei, et al.
Publicado: (2026)

Crisp Attention: Regularizing Transformers via Structured Sparsity
por: Gandhi, Sagar, et al.
Publicado: (2025)

Efficient Sparse Attention needs Adaptive Token Release
por: Zhang, Chaoran, et al.
Publicado: (2024)

Accelerating Prefilling via Decoding-time Contribution Sparsity
por: He, Zhiyuan, et al.
Publicado: (2025)

STS: Efficient Sparse Attention with Speculative Token Sparsity
por: Xu, Ceyu, et al.
Publicado: (2026)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
por: Qiu, Zihan, et al.
Publicado: (2025)

CMD: a framework for Context-aware Model self-Detoxification
por: Tang, Zecheng, et al.
Publicado: (2023)

Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models
por: Xiang, Yang, et al.
Publicado: (2025)

When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoning
por: Xiang, Yang, et al.
Publicado: (2026)

MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
por: Zhou, Keyan, et al.
Publicado: (2025)

Post-Training Sparse Attention with Double Sparsity
por: Yang, Shuo, et al.
Publicado: (2024)

OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning
por: Qiao, Dan, et al.
Publicado: (2024)

Sirius: Contextual Sparsity with Correction for Efficient LLMs
por: Zhou, Yang, et al.
Publicado: (2024)

ART: Attention Run-time Termination for Efficient Large Language Model Decoding
por: Qiu, Chen, et al.
Publicado: (2026)

Attention Condensation via Sparsity Induced Regularized Training
por: Sason, Eli, et al.
Publicado: (2025)

HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference
por: Ai, Xuan, et al.
Publicado: (2026)

AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
por: Yang, Leixin, et al.
Publicado: (2023)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
por: Zhao, Bowen, et al.
Publicado: (2024)

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
por: Yuan, Jiayi, et al.
Publicado: (2025)

LongFlow: Efficient KV Cache Compression for Reasoning Models
por: Su, Yi, et al.
Publicado: (2026)

OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure
por: Wang, Jikai, et al.
Publicado: (2024)

Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
por: Tian, Zhenxu, et al.
Publicado: (2026)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
por: Tang, Yehui, et al.
Publicado: (2025)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
por: Tang, Jiaming, et al.
Publicado: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
por: Yang, Songlin, et al.
Publicado: (2023)

Parallel Loop Transformer for Efficient Test-Time Computation Scaling
por: Wu, Bohong, et al.
Publicado: (2025)

MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models
por: Tang, Zecheng, et al.
Publicado: (2026)