Cover Image

Attention Sinks in Massively Multilingual Neural Machine Translation:Discovery, Analysis, and Mitigation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mutisya, Hillary, Mugane, John
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering
by: Mutisya, Hillary, et al.
Published: (2026)

Neural Recovery of Historical Lexical Structure in Bantu Languages from Modern Data
by: Mutisya, Hillary, et al.
Published: (2026)

The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages
by: Mutisya, Hillary, et al.
Published: (2026)

Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
by: Mutisya, Hillary, et al.
Published: (2026)

Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models
by: Mohammadshahi, Alireza, et al.
Published: (2023)

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
by: Son, Seungwoo, et al.
Published: (2024)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
by: Fu, Zizhuo, et al.
Published: (2026)

Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model
by: García-Romero, Cristian, et al.
Published: (2025)

On the Existence and Behavior of Secondary Attention Sinks
by: Wong, Jeffrey T. H., et al.
Published: (2025)

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
by: Binkowski, Jakub, et al.
Published: (2026)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
by: Shang, Bingqi, et al.
Published: (2025)

Self-Vocabularizing Training for Neural Machine Translation
by: Lin, Pin-Jie, et al.
Published: (2025)

Conditioning LLMs with Emotion in Neural Machine Translation
by: Brazier, Charles, et al.
Published: (2024)

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
by: Yu, Zhongzhi, et al.
Published: (2024)

Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation
by: Wu, Di, et al.
Published: (2023)

When Attention Sink Emerges in Language Models: An Empirical View
by: Gu, Xiangming, et al.
Published: (2024)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
by: Zhang, Stephen, et al.
Published: (2025)

Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective
by: Bai, Xueying, et al.
Published: (2024)

Enhanced Structured State Space Models via Grouped FIR Filtering and Attention Sink Mechanisms
by: Meng, Tian, et al.
Published: (2024)

Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation
by: Meng, Yan, et al.
Published: (2024)

To Label or Not to Label: Hybrid Active Learning for Neural Machine Translation
by: Azeemi, Abdul Hameed, et al.
Published: (2024)

Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing
by: Wang, Weichuan, et al.
Published: (2024)

Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine Translation
by: Emezue, Chris
Published: (2024)

Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
by: Mathewson, Kyle Elliott
Published: (2026)

An In-depth Walkthrough on Evolution of Neural Machine Translation
by: Jagtap, Rohan, et al.
Published: (2020)

An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation
by: Supryadi, et al.
Published: (2024)

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
by: Sutawika, Lintang, et al.
Published: (2026)

How Multilingual Are Large Language Models Fine-Tuned for Translation?
by: Richburg, Aquia, et al.
Published: (2024)

Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation
by: Heo, DongNyeong, et al.
Published: (2023)

From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation
by: Marashian, Ali, et al.
Published: (2024)

The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translation
by: Tokarchuk, Evgeniia, et al.
Published: (2023)

On Translating Technical Terminology: A Translation Workflow for Machine-Translated Acronyms
by: Yue, Richard, et al.
Published: (2024)

LOLA -- An Open-Source Massively Multilingual Large Language Model
by: Srivastava, Nikit, et al.
Published: (2024)

Curated Datasets and Neural Models for Machine Translation of Informal Registers between Mayan and Spanish Vernaculars
by: Lou, Andrés, et al.
Published: (2024)

GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
by: Hu, Yuchen, et al.
Published: (2024)

Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models
by: Mąka, Paweł, et al.
Published: (2024)

Self-generated Replay Memories for Continual Neural Machine Translation
by: Resta, Michele, et al.
Published: (2024)

Integrating Pre-trained Language Model into Neural Machine Translation
by: Hwang, Soon-Jae, et al.
Published: (2023)

Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation
by: Spravil, Julian, et al.
Published: (2025)