Image de couverture de livre

TransMLA: Multi-Head Latent Attention Is All You Need

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Meng, Fanxu, Tang, Pingzhi, Tang, Xiaojuan, Yao, Zengwei, Sun, Xing, Zhang, Muhan
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
par: Tang, Xiaojuan, et autres
Publié: (2025)

CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
par: Meng, Fanxu, et autres
Publié: (2024)

Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation
par: Tang, Pingzhi, et autres
Publié: (2026)

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
par: Meng, Fanxu
Publié: (2026)

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
par: Meng, Fanxu, et autres
Publié: (2024)

Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation
par: Tang, Pingzhi, et autres
Publié: (2026)

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
par: Xu, Yufei, et autres
Publié: (2026)

HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
par: Wang, Yiding, et autres
Publié: (2025)

Attention is All You Need Until You Need Retention
par: Yaslioglu, M. Murat
Publié: (2025)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
par: Zhou, Ruijie, et autres
Publié: (2026)

Tensor Product Attention Is All You Need
par: Zhang, Yifan, et autres
Publié: (2025)

Element-wise Attention Is All You Need
par: Feng, Guoxin
Publié: (2025)

RulE: Knowledge Graph Reasoning with Rule Embedding
par: Tang, Xiaojuan, et autres
Publié: (2022)

Superiority of Multi-Head Attention in In-Context Linear Regression
par: Cui, Yingqian, et autres
Publié: (2024)

MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models
par: Fan, Xiaoran, et autres
Publié: (2026)

Not All Attention Heads Are What You Need: Refining CLIP's Image Representation with Attention Ablation
par: Lin, Feng, et autres
Publié: (2025)

Attention Smoothing Is All You Need For Unlearning
par: Zade, Saleh Zare, et autres
Publié: (2026)

Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need
par: Xue, Runzhen, et autres
Publié: (2024)

Context is All You Need
par: Delanois, Jean Erik, et autres
Publié: (2026)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
par: Li, Yanan, et autres
Publié: (2025)

RecurFormer: Not All Transformer Heads Need Self-Attention
par: Yan, Ruiqing, et autres
Publié: (2024)

Attention Is All You Need for KV Cache in Diffusion LLMs
par: Nguyen-Tri, Quan, et autres
Publié: (2025)

Attention Is Not What You Need
par: Chong, Zhang
Publié: (2025)

TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
par: Yüzügüler, Ahmet Caner, et autres
Publié: (2025)

Exploitation Is All You Need... for Exploration
par: Rentschler, Micah, et autres
Publié: (2025)

A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals
par: Liu, Grace, et autres
Publié: (2024)

All You Need Is Synthetic Task Augmentation
par: Godin, Guillaume
Publié: (2025)

Is Diversity All You Need for Scalable Robotic Manipulation?
par: Shi, Modi, et autres
Publié: (2025)

More Agents Is All You Need
par: Li, Junyou, et autres
Publié: (2024)

CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention
par: Zhou, Zhongzhu, et autres
Publié: (2026)

Capabilities Ain't All You Need: Measuring Propensities in AI
par: Romero-Alvarado, Daniel, et autres
Publié: (2026)

DB2-TransF: All You Need Is Learnable Daubechies Wavelets for Time Series Forecasting
par: Gupta, Moulik, et autres
Publié: (2025)

Transduction is All You Need for Structured Data Workflows
par: Gliozzo, Alfio, et autres
Publié: (2025)

FlashMLA-ETAP: Efficient Transpose Attention Pipeline for Accelerating MLA Inference on NVIDIA H20 GPUs
par: Dege, Pengcuo, et autres
Publié: (2025)

What Matters in Transformers? Not All Attention is Needed
par: He, Shwai, et autres
Publié: (2024)

Cooperation Is All You Need
par: Adeel, Ahsan, et autres
Publié: (2023)

The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
par: Qasim, Kaleem Ullah, et autres
Publié: (2026)

Mars: Situated Inductive Reasoning in an Open-World Environment
par: Tang, Xiaojuan, et autres
Publié: (2024)

HDL-GPT: High-Quality HDL is All You Need
par: Kumar, Bhuvnesh, et autres
Publié: (2024)

OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agents
par: Zhou, Yuhang, et autres
Publié: (2026)