Cover Image

Knocking-Heads Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Zhanchao, Chen, Xiaodong, Chen, Haoxing, Lan, Zhenzhong, Li, Jianguo
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DND: Boosting Large Language Models with Dynamic Nested Depth
by: Chen, Tieyuan, et al.
Published: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

Value Residual Learning
by: Zhou, Zhanchao, et al.
Published: (2024)

QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution
by: Chen, Zijie, et al.
Published: (2024)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
by: Song, Chiyu, et al.
Published: (2023)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
by: Hu, Xiang, et al.
Published: (2025)

Inclusion Arena: An Open Platform for Evaluating Large Foundation Models with Real-World Apps
by: Wang, Kangyu, et al.
Published: (2025)

ConceptPsy:A Benchmark Suite with Conceptual Comprehensiveness in Psychology
by: Zhang, Junlei, et al.
Published: (2023)

PsyGUARD: An Automated System for Suicide Detection and Risk Assessment in Psychological Counseling
by: Qiu, Huachuan, et al.
Published: (2024)

PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling
by: Qiu, Huachuan, et al.
Published: (2026)

RECAP: Resistance Capture in Text-based Mental Health Counseling with Large Language Models
by: Li, Anqi, et al.
Published: (2026)

PsyChat: A Client-Centric Dialogue System for Mental Health Support
by: Qiu, Huachuan, et al.
Published: (2023)

CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
by: Wang, Kangyu, et al.
Published: (2025)

Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic
by: Yan, Yang, et al.
Published: (2025)

Improving Data and Reward Design for Scientific Reasoning in Large Language Models
by: Chen, Zijie, et al.
Published: (2026)

Enhancing Chest X-ray Classification through Knowledge Injection in Cross-Modality Learning
by: Yan, Yang, et al.
Published: (2025)

ZigzagAttention: Efficient Long-Context Inference with Exclusive Retrieval and Streaming Heads
by: Liu, Zhuorui, et al.
Published: (2025)

Debiasing LLMs by Masking Unfairness-Driving Attention Heads
by: Han, Tingxu, et al.
Published: (2025)

Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs
by: Li, Anqi, et al.
Published: (2026)

Facilitating Pornographic Text Detection for Open-Domain Dialogue Systems via Knowledge Distillation of Large Language Models
by: Qiu, Huachuan, et al.
Published: (2024)

Facilitating NSFW Text Detection in Open-Domain Dialogue Systems via Knowledge Distillation
by: Qiu, Huachuan, et al.
Published: (2023)

SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Support
by: Qiu, Huachuan, et al.
Published: (2023)

BOSCH: Black-Box Binary Optimization for Short-Context Attention-Head Selection in LLMs
by: Ghaddar, Abbas, et al.
Published: (2026)

Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads
by: He, Xingyang, et al.
Published: (2025)

Multi-Head Attention Driven Dynamic Visual-Semantic Embedding for Enhanced Image-Text Matching
by: Chen, Wenjing
Published: (2024)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
by: Lu, Yi, et al.
Published: (2024)

Integrating Multi-Head Convolutional Encoders with Cross-Attention for Improved SPARQL Query Translation
by: Chen, Yi-Hui, et al.
Published: (2024)

Attention Heads of Large Language Models: A Survey
by: Zheng, Zifan, et al.
Published: (2024)

S2-Attention: Hardware-Aware Context Sharding Among Attention Heads
by: Lin, Xihui, et al.
Published: (2024)

CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models
by: Li, Anqi, et al.
Published: (2026)

Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs
by: Li, Anqi, et al.
Published: (2024)

Predicting the Big Five Personality Traits in Chinese Counselling Dialogues Using Large Language Models
by: Yan, Yang, et al.
Published: (2024)

Interpreting Transformers Through Attention Head Intervention
by: Kadem, Mason, et al.
Published: (2026)

Inferring Functionality of Attention Heads from their Parameters
by: Elhelo, Amit, et al.
Published: (2024)

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA
by: Jin, Qingyun, et al.
Published: (2024)

Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
by: He, Zhihao, et al.
Published: (2024)

Head-wise Shareable Attention for Large Language Models
by: Cao, Zouying, et al.
Published: (2024)

Improving Transformers with Dynamically Composable Multi-Head Attention
by: Xiao, Da, et al.
Published: (2024)

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
by: Chen, Yilong, et al.
Published: (2024)

AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection
by: Hua, Kai, et al.
Published: (2025)