Image de couverture de livre

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Shao, Jiandong, Tang, Raphael, Zhang, Crystina, Sevegnani, Karin, Stenetorp, Pontus, Yang, Jianfei, Lu, Yao
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Multilingual Language Model Pretraining using Machine-translated Data
par: Wang, Jiayi, et autres
Publié: (2025)

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
par: Wang, Jiayi, et autres
Publié: (2024)

Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style LLM Evaluation
par: Tang, Raphael, et autres
Publié: (2025)

Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models
par: Madaan, Lovish, et autres
Publié: (2024)

Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation
par: Lu, Yao, et autres
Publié: (2023)

Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
par: Shao, Jiandong, et autres
Publié: (2025)

Gender-specific Machine Translation with Large Language Models
par: Sánchez, Eduardo, et autres
Publié: (2023)

Improving Language Plasticity via Pretraining with Active Forgetting
par: Chen, Yihong, et autres
Publié: (2023)

Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles
par: Trhlik, Filip, et autres
Publié: (2024)

DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining
par: Yan, Yutong, et autres
Publié: (2026)

AfroBench: How Good are Large Language Models on African Languages?
par: Ojo, Jessica, et autres
Publié: (2023)

Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?
par: Zhang, Crystina, et autres
Publié: (2024)

Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation
par: Tang, Raphael, et autres
Publié: (2024)

Using Natural Language Explanations to Improve Robustness of In-context Learning
par: He, Xuanli, et autres
Publié: (2023)

Jet Expansions of Residual Computation
par: Chen, Yihong, et autres
Publié: (2024)

Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models
par: Benkirane, Kenza, et autres
Publié: (2024)

Multilingual Pretraining for Pixel Language Models
par: Kesen, Ilker, et autres
Publié: (2025)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
par: Guo, Ping, et autres
Publié: (2025)

Disentangling Language and Culture for Evaluating Multilingual Large Language Models
par: Ying, Jiahao, et autres
Publié: (2025)

Factual Consistency of Multilingual Pretrained Language Models
par: Fierro, Constanza, et autres
Publié: (2022)

Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources
par: Li, Zihao, et autres
Publié: (2025)

Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models
par: Lu, Hongyuan, et autres
Publié: (2024)

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives
par: Li, Zihao, et autres
Publié: (2024)

Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models
par: Tang, Tianyi, et autres
Publié: (2024)

A Comparative Study of Translation Bias and Accuracy in Multilingual Large Language Models for Cross-Language Claim Verification
par: Singhal, Aryan, et autres
Publié: (2024)

Data Mixing for Large Language Models Pretraining: A Survey and Outlook
par: Chen, Zhuo, et autres
Publié: (2026)

AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment
par: Bu, Mengyu, et autres
Publié: (2025)

Multilingual Large Language Models and Curse of Multilinguality
par: Gurgurov, Daniil, et autres
Publié: (2024)

Multilingual Contextualization of Large Language Models for Document-Level Machine Translation
par: Ramos, Miguel Moura, et autres
Publié: (2025)

LangSAMP: Language-Script Aware Multilingual Pretraining
par: Liu, Yihong, et autres
Publié: (2024)

Language Surgery in Multilingual Large Language Models
par: Lopo, Joanito Agili, et autres
Publié: (2025)

Multilingual Synopses of Movie Narratives: A Dataset for Vision-Language Story Understanding
par: Sun, Yidan, et autres
Publié: (2024)

Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
par: Son, Guijin, et autres
Publié: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
par: Foroutan, Negar, et autres
Publié: (2025)

Pruning Multilingual Large Language Models for Multilingual Inference
par: Kim, Hwichan, et autres
Publié: (2024)

Multilingual Jailbreak Challenges in Large Language Models
par: Deng, Yue, et autres
Publié: (2023)

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints
par: Song, Ran, et autres
Publié: (2024)

Progressive Residual Warmup for Language Model Pretraining
par: Chen, Tianhao, et autres
Publié: (2026)

Quantifying Language Disparities in Multilingual Large Language Models
par: Hu, Songbo, et autres
Publié: (2025)

Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind
par: Zeng, Hongchuan, et autres
Publié: (2024)