Image de couverture de livre

Multilingual Language Model Pretraining using Machine-translated Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Jiayi, Lu, Yao, Weber, Maurice, Ryabinin, Max, Adelani, David, Chen, Yihong, Tang, Raphael, Stenetorp, Pontus
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
par: Wang, Jiayi, et autres
Publié: (2024)

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
par: Shao, Jiandong, et autres
Publié: (2026)

Improving Language Plasticity via Pretraining with Active Forgetting
par: Chen, Yihong, et autres
Publié: (2023)

Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation
par: Lu, Yao, et autres
Publié: (2023)

Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models
par: Benkirane, Kenza, et autres
Publié: (2024)

Jet Expansions of Residual Computation
par: Chen, Yihong, et autres
Publié: (2024)

Warmup Generations: A Task-Agnostic Approach for Guiding Sequence-to-Sequence Learning with Unsupervised Initial State Generation
par: Li, Senyu, et autres
Publié: (2025)

AfroBench: How Good are Large Language Models on African Languages?
par: Ojo, Jessica, et autres
Publié: (2023)

Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style LLM Evaluation
par: Tang, Raphael, et autres
Publié: (2025)

Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles
par: Trhlik, Filip, et autres
Publié: (2024)

Gender-specific Machine Translation with Large Language Models
par: Sánchez, Eduardo, et autres
Publié: (2023)

Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models
par: Madaan, Lovish, et autres
Publié: (2024)

MINERS: Multilingual Language Models as Semantic Retrievers
par: Winata, Genta Indra, et autres
Publié: (2024)

LangSAMP: Language-Script Aware Multilingual Pretraining
par: Liu, Yihong, et autres
Publié: (2024)

TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data
par: Liu, Yihong, et autres
Publié: (2024)

Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation
par: Tang, Raphael, et autres
Publié: (2024)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
par: Ali, Mehdi, et autres
Publié: (2025)

ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective
par: Chen, Yihong, et autres
Publié: (2022)

How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective
par: Qiu, Xinchi, et autres
Publié: (2024)

SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?
par: Li, Senyu, et autres
Publié: (2025)

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models
par: Liu, Yihong, et autres
Publié: (2024)

Using Natural Language Explanations to Improve Robustness of In-context Learning
par: He, Xuanli, et autres
Publié: (2023)

Translate, then Detect: Leveraging Machine Translation for Cross-Lingual Toxicity Classification
par: Bell, Samuel J., et autres
Publié: (2025)

DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining
par: Yan, Yutong, et autres
Publié: (2026)

Tracing Multilingual Factual Knowledge Acquisition in Pretraining
par: Liu, Yihong, et autres
Publié: (2025)

ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Model
par: Quinjica, Osvaldo Luamba, et autres
Publié: (2024)

Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
par: Schmidt, Fabian David, et autres
Publié: (2025)

An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages
par: Lu, Yinhan, et autres
Publié: (2026)

Improving Multilingual Math Reasoning for African Languages
par: Ogundepo, Odunayo, et autres
Publié: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
par: Foroutan, Negar, et autres
Publié: (2025)

Multilinguality as Sense Adaptation
par: Cruz, Jan Christian Blaise, et autres
Publié: (2026)

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
par: Liu, Yihong, et autres
Publié: (2023)

Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements
par: Voronov, Anton, et autres
Publié: (2024)

Multilingual Pretraining for Pixel Language Models
par: Kesen, Ilker, et autres
Publié: (2025)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
par: Guo, Ping, et autres
Publié: (2025)

Evaluating Contextually Mediated Factual Recall in Multilingual Large Language Models
par: Liu, Yihong, et autres
Publié: (2026)

Factual Consistency of Multilingual Pretrained Language Models
par: Fierro, Constanza, et autres
Publié: (2022)

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives
par: Li, Zihao, et autres
Publié: (2024)

AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages
par: Uemura, Kosei, et autres
Publié: (2025)

Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs
par: Adelani, David Ifeoluwa, et autres
Publié: (2024)