Imagen de Portada

GTA: Grouped-head latenT Attention

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Sun, Luoyang, Deng, Cheng, Jiang, Jiwen, Wu, Xinjian, Zhang, Haifeng, Chen, Lei, Ni, Lionel, Wang, Jun
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

GTA: A Benchmark for General Tool Agents
por: Wang, Jize, et al.
Publicado: (2024)

PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing
por: Deng, Cheng, et al.
Publicado: (2025)

GTA: Generating Long-Horizon Tasks for Web Agents at Scale
por: Huang, Tenghao, et al.
Publicado: (2026)

GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows
por: Wang, Jize, et al.
Publicado: (2026)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
por: Peng, Runyu, et al.
Publicado: (2026)

MARRO: Multi-headed Attention for Rhetorical Role Labeling in Legal Documents
por: Bambroo, Purbid, et al.
Publicado: (2025)

Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation
por: Li, Shuaiyi, et al.
Publicado: (2026)

Pay Attention to What You Need
por: Gao, Yifei, et al.
Publicado: (2023)

More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection
por: Sun, Runze, et al.
Publicado: (2026)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
por: Chen, Yingfa, et al.
Publicado: (2025)

Weighted Grouped Query Attention in Transformers
por: Chinnakonduru, Sai Sena, et al.
Publicado: (2024)

GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification
por: Wen, Ximing, et al.
Publicado: (2024)

GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion
por: Liu, Tongxuan, et al.
Publicado: (2024)

Activation-aware Probe-Query: Effective Key-Value Retrieval for Long-Context LLMs Inference
por: Xiao, Qingfa, et al.
Publicado: (2025)

Neural Attention Search
por: Deng, Difan, et al.
Publicado: (2025)

TacticCraft: Natural Language-Driven Tactical Adaptation for StarCraft II
por: Ma, Weiyu, et al.
Publicado: (2025)

OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
por: Wang, Jun, et al.
Publicado: (2024)

Mixture-of-Depths Attention
por: Zhu, Lianghui, et al.
Publicado: (2026)

Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference
por: Tang, Yaohua, et al.
Publicado: (2025)

Improving Retrieval Augmented Language Model with Self-Reasoning
por: Xia, Yuan, et al.
Publicado: (2024)

Token-level Direct Preference Optimization
por: Zeng, Yongcheng, et al.
Publicado: (2024)

Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learning
por: Wu, John, et al.
Publicado: (2024)

HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models
por: Jiang, Xinyan, et al.
Publicado: (2025)

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
por: Li, Chong, et al.
Publicado: (2025)

MatryoshkaThinking: Recursive Test-Time Scaling Enables Efficient Reasoning
por: Chen, Hongwei, et al.
Publicado: (2025)

LinearARD: Linear-Memory Attention Distillation for RoPE Restoration
por: Yang, Ning, et al.
Publicado: (2026)

Towards Better Text-to-Image Generation Alignment via Attention Modulation
por: Wu, Yihang, et al.
Publicado: (2024)

Efficient Context Scaling with LongCat ZigZag Attention
por: Zhang, Chen, et al.
Publicado: (2025)

MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens
por: Chen, Yu, et al.
Publicado: (2026)

Native Hybrid Attention for Efficient Sequence Modeling
por: Du, Jusen, et al.
Publicado: (2025)

Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation
por: Lin, Minhua, et al.
Publicado: (2024)

Yuan 2.0-M32: Mixture of Experts with Attention Router
por: Wu, Shaohua, et al.
Publicado: (2024)

Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach
por: Jiang, Zhouyu, et al.
Publicado: (2024)

Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models
por: Gu, Leijiang, et al.
Publicado: (2026)

Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach
por: Aboagye, Prince, et al.
Publicado: (2024)

Attention Needs to Focus: A Unified Perspective on Attention Allocation
por: Fu, Zichuan, et al.
Publicado: (2026)

MC-GTA: Metric-Constrained Model-Based Clustering using Goodness-of-fit Tests with Autocorrelations
por: Wang, Zhangyu, et al.
Publicado: (2024)

Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access
por: Hu, Xiang, et al.
Publicado: (2025)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
por: Gao, Yizhao, et al.
Publicado: (2026)

Sim2Sea: Sim-to-Real Policy Transfer for Maritime Vessel Navigation in Congested Waters
por: Cui, Xinyu, et al.
Publicado: (2026)