Cover Image

Self-Selected Attention Span for Accelerating Large Language Model Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jin, Tian, Yazar, Wanzin, Xu, Zifei, Sharify, Sayeh, Wang, Xin
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Post Training Quantization of Large Language Models with Microscaling Formats
by: Sharify, Sayeh, et al.
Published: (2024)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Early Attentive Sparsification Accelerates Neural Speech Transcription
by: Xu, Zifei, et al.
Published: (2025)

Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
by: Xu, Zifei, et al.
Published: (2024)

MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference
by: Xu, Zifei, et al.
Published: (2026)

ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
by: Saxena, Utkarsh, et al.
Published: (2024)

DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration
by: Zhang, Hanzhi, et al.
Published: (2025)

AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
by: Gu, Yifeng, et al.
Published: (2025)

Attention-Aligned Reasoning for Large Language Models
by: Zhang, Hongxiang, et al.
Published: (2025)

Exploring the Performance of Large Language Models on Subjective Span Identification Tasks
by: Dmonte, Alphaeus, et al.
Published: (2026)

Focusing on Language: Revealing and Exploiting Language Attention Heads in Multilingual Large Language Models
by: Liu, Xin, et al.
Published: (2025)

Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models
by: Liu, Qiang, et al.
Published: (2025)

Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference
by: Tang, Yaohua, et al.
Published: (2025)

Scaling Bidirectional Spans and Span Violations in Attention Mechanism
by: Kim, Jongwook, et al.
Published: (2025)

Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens
by: Zeng, Ziqian, et al.
Published: (2024)

HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models
by: Yang, Ze, et al.
Published: (2024)

S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models
by: He, Tao, et al.
Published: (2025)

A Survey on Efficient Inference for Large Language Models
by: Zhou, Zixuan, et al.
Published: (2024)

Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
by: Chen, Yushuo, et al.
Published: (2024)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
by: Li, Haoling, et al.
Published: (2024)

ContextGuard: Structured Self-Auditing for Context Learning in Language Models
by: Jin, Hongbo, et al.
Published: (2026)

S^3cMath: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners
by: Yan, Yuchen, et al.
Published: (2024)

Plato: Plan to Efficiently Decode for Large Language Model Inference
by: Jin, Shuowei, et al.
Published: (2024)

Quantum-Inspired Self-Attention in a Large Language Model
by: Kuznetsov, Nikita, et al.
Published: (2026)

Large Language Models and Causal Inference in Collaboration: A Survey
by: Liu, Xiaoyu, et al.
Published: (2024)

Accelerating Large Language Model Reasoning via Speculative Search
by: Wang, Zhihai, et al.
Published: (2025)

Cross-Attention Watermarking of Large Language Models
by: Baldassini, Folco Bertini, et al.
Published: (2024)

A Survey on Large Language Models for Mathematical Reasoning
by: Wang, Peng-Yuan, et al.
Published: (2025)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
by: Zhu, Qianchao, et al.
Published: (2024)

A Survey on Self-Evolution of Large Language Models
by: Tao, Zhengwei, et al.
Published: (2024)

Vision Token Reduction via Attention-Driven Self-Compression for Efficient Multimodal Large Language Models
by: Deniz, Omer Faruk, et al.
Published: (2026)

Efficient Attention Mechanisms for Large Language Models: A Survey
by: Sun, Yutao, et al.
Published: (2025)

Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models
by: Xu, Ningyu, et al.
Published: (2026)

CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models
by: Wagner, Eitan, et al.
Published: (2024)

HAVE: Head-Adaptive Gating and ValuE Calibration for Hallucination Mitigation in Large Language Models
by: Tong, Xin, et al.
Published: (2025)

Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model
by: Tian, Yuanhe, et al.
Published: (2025)

Progressively Label Enhancement for Large Language Model Alignment
by: Liu, Biao, et al.
Published: (2024)

Dynamic Compressing Prompts for Efficient Inference of Large Language Models
by: Hu, Jinwu, et al.
Published: (2025)

Hypothesis Testing Prompting Improves Deductive Reasoning in Large Language Models
by: Li, Yitian, et al.
Published: (2024)

Interpretable Differential Diagnosis with Dual-Inference Large Language Models
by: Zhou, Shuang, et al.
Published: (2024)