Image de couverture de livre

Attention Sinks in Massively Multilingual Neural Machine Translation:Discovery, Analysis, and Mitigation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Mutisya, Hillary, Mugane, John
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering
par: Mutisya, Hillary, et autres
Publié: (2026)

Neural Recovery of Historical Lexical Structure in Bantu Languages from Modern Data
par: Mutisya, Hillary, et autres
Publié: (2026)

The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages
par: Mutisya, Hillary, et autres
Publié: (2026)

Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
par: Mutisya, Hillary, et autres
Publié: (2026)

Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models
par: Mohammadshahi, Alireza, et autres
Publié: (2023)

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
par: Son, Seungwoo, et autres
Publié: (2024)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
par: Fu, Zizhuo, et autres
Publié: (2026)

Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model
par: García-Romero, Cristian, et autres
Publié: (2025)

On the Existence and Behavior of Secondary Attention Sinks
par: Wong, Jeffrey T. H., et autres
Publié: (2025)

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
par: Binkowski, Jakub, et autres
Publié: (2026)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
par: Peng, Runyu, et autres
Publié: (2026)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
par: Shang, Bingqi, et autres
Publié: (2025)

Self-Vocabularizing Training for Neural Machine Translation
par: Lin, Pin-Jie, et autres
Publié: (2025)

Conditioning LLMs with Emotion in Neural Machine Translation
par: Brazier, Charles, et autres
Publié: (2024)

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
par: Yu, Zhongzhi, et autres
Publié: (2024)

Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation
par: Wu, Di, et autres
Publié: (2023)

When Attention Sink Emerges in Language Models: An Empirical View
par: Gu, Xiangming, et autres
Publié: (2024)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
par: Zhang, Stephen, et autres
Publié: (2025)

Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective
par: Bai, Xueying, et autres
Publié: (2024)

Enhanced Structured State Space Models via Grouped FIR Filtering and Attention Sink Mechanisms
par: Meng, Tian, et autres
Publié: (2024)

Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation
par: Meng, Yan, et autres
Publié: (2024)

To Label or Not to Label: Hybrid Active Learning for Neural Machine Translation
par: Azeemi, Abdul Hameed, et autres
Publié: (2024)

Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing
par: Wang, Weichuan, et autres
Publié: (2024)

Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine Translation
par: Emezue, Chris
Publié: (2024)

Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
par: Mathewson, Kyle Elliott
Publié: (2026)

An In-depth Walkthrough on Evolution of Neural Machine Translation
par: Jagtap, Rohan, et autres
Publié: (2020)

An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation
par: Supryadi, et autres
Publié: (2024)

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
par: Sutawika, Lintang, et autres
Publié: (2026)

How Multilingual Are Large Language Models Fine-Tuned for Translation?
par: Richburg, Aquia, et autres
Publié: (2024)

Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation
par: Heo, DongNyeong, et autres
Publié: (2023)

From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation
par: Marashian, Ali, et autres
Publié: (2024)

The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translation
par: Tokarchuk, Evgeniia, et autres
Publié: (2023)

On Translating Technical Terminology: A Translation Workflow for Machine-Translated Acronyms
par: Yue, Richard, et autres
Publié: (2024)

LOLA -- An Open-Source Massively Multilingual Large Language Model
par: Srivastava, Nikit, et autres
Publié: (2024)

Curated Datasets and Neural Models for Machine Translation of Informal Registers between Mayan and Spanish Vernaculars
par: Lou, Andrés, et autres
Publié: (2024)

GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
par: Hu, Yuchen, et autres
Publié: (2024)

Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models
par: Mąka, Paweł, et autres
Publié: (2024)

Self-generated Replay Memories for Continual Neural Machine Translation
par: Resta, Michele, et autres
Publié: (2024)

Integrating Pre-trained Language Model into Neural Machine Translation
par: Hwang, Soon-Jae, et autres
Publié: (2023)

Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation
par: Spravil, Julian, et autres
Publié: (2025)