Cover Image

MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fan, Xiaoran, Sun, Zhichao, Ji, Tao, Shen, Lixing, Gui, Tao
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
by: Ji, Tao, et al.
Published: (2025)

Hardware-Centric Analysis of DeepSeek's Multi-Head Latent Attention
by: Geens, Robin, et al.
Published: (2025)

DeepSeek-OCR: Contexts Optical Compression
by: Wei, Haoran, et al.
Published: (2025)

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
by: Wu, Zhiyu, et al.
Published: (2024)

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
by: DeepSeek-AI, et al.
Published: (2024)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
by: Lu, Yi, et al.
Published: (2024)

DeepSeek-OCR 2: Visual Causal Flow
by: Wei, Haoran, et al.
Published: (2026)

EG-MLA: Embedding-Gated Multi-head Latent Attention for Scalable and Efficient LLMs
by: Cai, Zhengge, et al.
Published: (2025)

DeepSeek-V3 Technical Report
by: DeepSeek-AI, et al.
Published: (2024)

An evaluation of DeepSeek Models in Biomedical Natural Language Processing
by: Zhan, Zaifu, et al.
Published: (2025)

Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek
by: Xiao, Dongfu, et al.
Published: (2025)

Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings
by: Ying, Zonghao, et al.
Published: (2025)

Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA
by: Jin, Qingyun, et al.
Published: (2024)

A Comparison of DeepSeek and Other LLMs
by: Gao, Tianchen, et al.
Published: (2025)

DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
by: DeepSeek-AI, et al.
Published: (2025)

Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts
by: Zhang, Wenjing, et al.
Published: (2025)

Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek
by: Qiu, Peiran, et al.
Published: (2025)

Can DeepSeek Reason Like a Surgeon? An Empirical Evaluation for Vision-Language Understanding in Robotic-Assisted Surgery
by: Ma, Boyi, et al.
Published: (2025)

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
by: DeepSeek-AI, et al.
Published: (2024)

TransMLA: Multi-Head Latent Attention Is All You Need
by: Meng, Fanxu, et al.
Published: (2025)

DeepSeek reshaping healthcare in China's tertiary hospitals
by: Chen, Jishizhan, et al.
Published: (2025)

Safety Evaluation of DeepSeek Models in Chinese Contexts
by: Zhang, Wenjing, et al.
Published: (2025)

ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp
by: Chen, Xinhang, et al.
Published: (2025)

The Role of Entropy in Visual Grounding: Analysis and Optimization
by: Li, Shuo, et al.
Published: (2025)

X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression
by: Li, Guihong, et al.
Published: (2025)

Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR
by: Liang, Yunhao, et al.
Published: (2026)

From ChatGPT to DeepSeek: Can LLMs Simulate Humanity?
by: Wang, Qian, et al.
Published: (2025)

Medical Reasoning in LLMs: An In-Depth Analysis of DeepSeek R1
by: Moell, Birger, et al.
Published: (2025)

DeepSeek performs better than other Large Language Models in Dental Cases
by: Zhang, Hexian, et al.
Published: (2025)

Latent Multi-Head Attention for Small Language Models
by: Mehta, Sushant, et al.
Published: (2025)

From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models
by: Zhang, Jue, et al.
Published: (2025)

DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning
by: Marjanović, Sara Vera, et al.
Published: (2025)

User Intent to Use DeepSeek for Healthcare Purposes and their Trust in the Large Language Model: Multinational Survey Study
by: Choudhury, Avishek, et al.
Published: (2025)

Can AI be a Teaching Partner? Evaluating ChatGPT, Gemini, and DeepSeek across Three Teaching Strategies
by: de Souza, Talita de Paula Cypriano, et al.
Published: (2026)

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations
by: Li, Shuo, et al.
Published: (2025)

DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural prompting
by: Luther, James, et al.
Published: (2025)

DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
by: Guo, Daya, et al.
Published: (2024)

R1dacted: Investigating Local Censorship in DeepSeek's R1 Language Model
by: Naseh, Ali, et al.
Published: (2025)

Benchmark-Driven Selection of AI: Evidence from DeepSeek-R1
by: Spelda, Petr, et al.
Published: (2025)

Output Length Effect on DeepSeek-R1's Safety in Forced Thinking
by: Li, Xuying, et al.
Published: (2025)