Cover Image

Less Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Lijie, Zhang, Zhihao, Jain, Arti, Cao, Shijie, Yuan, Baihong, Chen, Yiwei, Jia, Zhihao, Netravali, Ravi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
by: Yang, Lijie, et al.
Published: (2024)

SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
by: Pan, Rui, et al.
Published: (2025)

LIMO: Less is More for Reasoning
by: Ye, Yixin, et al.
Published: (2025)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
by: Zhuang, Xialie, et al.
Published: (2025)

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework
by: Zhang, Chenyuan, et al.
Published: (2026)

Accelerating Retrieval-Augmented Language Model Serving with Speculation
by: Zhang, Zhihao, et al.
Published: (2024)

Rectified Sparse Attention
by: Sun, Yutao, et al.
Published: (2025)

Geometry Guided Self-Consistency for Physical AI
by: Dai, Yinwei, et al.
Published: (2026)

SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
by: Gao, Yizhao, et al.
Published: (2024)

Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
by: Yang, Zhen, et al.
Published: (2025)

Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention
by: Zhan, Zhihao, et al.
Published: (2025)

LimRank: Less is More for Reasoning-Intensive Information Reranking
by: Song, Tingyu, et al.
Published: (2025)

A Unified Sparse Attention via Multi-Granularity Compression
by: Liu, Siran, et al.
Published: (2025)

Acting Less is Reasoning More! Teaching Model to Act Efficiently
by: Wang, Hongru, et al.
Published: (2025)

Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality
by: Hoang, Duy C., et al.
Published: (2024)

Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMs
by: Chen, Kang, et al.
Published: (2025)

When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
by: Zhao, Weixiang, et al.
Published: (2025)

Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance
by: Yu, Jiachen, et al.
Published: (2026)

Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining
by: Zhu, Jinchang, et al.
Published: (2026)

Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
by: Puvvada, Krishna C., et al.
Published: (2024)

Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
by: Liu, Jinzhe, et al.
Published: (2025)

Less Peaky and More Accurate CTC Forced Alignment by Label Priors
by: Huang, Ruizhe, et al.
Published: (2024)

METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
by: Ray, Siddhant, et al.
Published: (2024)

Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs
by: Banayeeanzade, Amin, et al.
Published: (2026)

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
by: Sui, Yi, et al.
Published: (2026)

Head-wise Shareable Attention for Large Language Models
by: Cao, Zouying, et al.
Published: (2024)

RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
by: Liu, Siran, et al.
Published: (2026)

Less is More: Compact Clue Selection for Efficient Retrieval-Augmented Generation Reasoning
by: Zhang, Qianchi, et al.
Published: (2025)

More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoning
by: Shafiei, Mohammadamin, et al.
Published: (2025)

Should We Attend More or Less? Modulating Attention for Fairness
by: Zayed, Abdelrahman, et al.
Published: (2023)

Soft Prompt Tuning for Cross-Lingual Transfer: When Less is More
by: Philippy, Fred, et al.
Published: (2024)

LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation
by: Yuan, Jiahao, et al.
Published: (2025)

SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning
by: Wen, Zhihao, et al.
Published: (2024)

Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking
by: Cheng, Xiaoxue, et al.
Published: (2025)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
by: Hu, Junhao, et al.
Published: (2026)

BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning
by: Gu, Jia-Chen, et al.
Published: (2025)

Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
by: Shrivastava, Vaishnavi, et al.
Published: (2025)

FASTTRACK: Fast and Accurate Fact Tracing for LLMs
by: Chen, Si, et al.
Published: (2024)

Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning
by: Yang, Haolin, et al.
Published: (2025)