Image de couverture de livre

MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zuhri, Zayd Muhammad Kawakibi, Adilazuarda, Muhammad Farid, Purwarianti, Ayu, Aji, Alham Fikri
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
par: Ananta, Moses, et autres
Publié: (2025)

Predicting the Order of Upcoming Tokens Improves Language Modeling
par: Zuhri, Zayd M. K., et autres
Publié: (2025)

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
par: Zuhri, Zayd M. K., et autres
Publié: (2025)

LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
par: Adilazuarda, Muhammad Farid, et autres
Publié: (2024)

Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation
par: Nur'aini, Khumaisa, et autres
Publié: (2026)

From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs
par: Adilazuarda, Muhammad Farid, et autres
Publié: (2025)

NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts
par: Adilazuarda, Muhammad Farid, et autres
Publié: (2025)

Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models
par: Susanto, Lucky, et autres
Publié: (2026)

PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues
par: Farhansyah, Mohammad Rifqi, et autres
Publié: (2026)

MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storage
par: He, Yongjun, et autres
Publié: (2025)

Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting
par: Mukherjee, Sagnik, et autres
Publié: (2024)

Crosslingual Reasoning through Test-Time Scaling
par: Yong, Zheng-Xin, et autres
Publié: (2025)

Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
par: Qorib, Muhammad Reza, et autres
Publié: (2024)

Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text
par: Adilazuarda, Muhammad Farid
Publié: (2023)

Towards Measuring and Modeling "Culture" in LLMs: A Survey
par: Adilazuarda, Muhammad Farid, et autres
Publié: (2024)

Daisy-TTS: Simulating Wider Spectrum of Emotions via Prosody Embedding Decomposition
par: Chevi, Rendi, et autres
Publié: (2024)

LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages
par: Aji, Alham Fikri, et autres
Publié: (2025)

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning
par: Attia, Ahmed, et autres
Publié: (2026)

Extracting General-use Transformers for Low-resource Languages via Knowledge Distillation
par: Cruz, Jan Christian Blaise, et autres
Publié: (2025)

The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation
par: Wibowo, Haryo Akbarianto, et autres
Publié: (2024)

Unveiling the Influence of Amplifying Language-Specific Neurons
par: Rahmanisa, Inaya, et autres
Publié: (2025)

CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections
par: Imam, Mohamed Fazli, et autres
Publié: (2024)

Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders
par: Ye, Mengyu, et autres
Publié: (2025)

Balanced Multi-Factor In-Context Learning for Multilingual Large Language Models
par: Kaneko, Masahiro, et autres
Publié: (2025)

Provable Robustness Against a Union of $\ell_0$ Adversarial Attacks
par: Hammoudeh, Zayd, et autres
Publié: (2023)

Training Data Influence Analysis and Estimation: A Survey
par: Hammoudeh, Zayd, et autres
Publié: (2022)

Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountability
par: Winata, Genta Indra, et autres
Publié: (2025)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
par: Brandon, William, et autres
Publié: (2024)

ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
par: Yang, David H., et autres
Publié: (2026)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
par: Musat, Tiberiu
Publié: (2024)

Data Laundering: Artificially Boosting Benchmark Results through Knowledge Distillation
par: Mansurov, Jonibek, et autres
Publié: (2024)

Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
par: Lyu, Chenyang, et autres
Publié: (2024)

Sense Representations Are Inducible Interfaces
par: Cruz, Jan Christian Blaise, et autres
Publié: (2026)

LLM Olympiad: Why Model Evaluation Needs a Sealed Exam
par: Cruz, Jan Christian Blaise, et autres
Publié: (2026)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
par: Devine, Peter, et autres
Publié: (2026)

LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers
par: Karmore, Aryan
Publié: (2026)

Interdomain Attention: Beyond Token-Level Key-Value Memory
par: Kiyohara, Naoki, et autres
Publié: (2026)

LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
par: Wu, Minghao, et autres
Publié: (2023)

PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration
par: Wu, Songhao, et autres
Publié: (2025)

Adaptive Head Budgeting for Efficient Multi-Head Attention
par: Faye, Bilal, et autres
Publié: (2026)