Cover Image

Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Alrashed, Sultan, Orabona, Francesco
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data
by: Alrashed, Sultan, et al.
Published: (2025)

SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs
by: Alrashed, Sultan
Published: (2024)

PolyMinHash: Efficient Area-Based MinHashing of Polygons for Approximate Nearest Neighbor Search
by: Subedi, Alima, et al.
Published: (2025)

Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models
by: Alrashed, Sultan, et al.
Published: (2024)

Real-Time Device Reach Forecasting Using HLL and MinHash Data Sketches
by: Muniyappa, Chandrashekar, et al.
Published: (2025)

Maintaining $k$-MinHash Signatures over Fully-Dynamic Data Streams with Recovery
by: Clementi, Andrea, et al.
Published: (2024)

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
by: Shao, Jiandong, et al.
Published: (2026)

Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources
by: Li, Zihao, et al.
Published: (2025)

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?
by: Ji, Shaoxiong, et al.
Published: (2024)

Multilingual JobBERT for Cross-Lingual Job Title Matching
by: Decorte, Jens-Joris, et al.
Published: (2025)

Relationships are Complicated! An Analysis of Relationships Between Datasets on the Web
by: Lin, Kate, et al.
Published: (2024)

Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection
by: Turki, Yassine, et al.
Published: (2026)

Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
by: Gao, Changjiang, et al.
Published: (2024)

Multilingual Topic Classification in X: Dataset and Analysis
by: Antypas, Dimosthenis, et al.
Published: (2024)

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
by: Wang, Jiayi, et al.
Published: (2024)

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
by: Lin, Peiqin, et al.
Published: (2023)

Tracing Multilingual Factual Knowledge Acquisition in Pretraining
by: Liu, Yihong, et al.
Published: (2025)

Multilingual Pretraining for Pixel Language Models
by: Kesen, Ilker, et al.
Published: (2025)

LangSAMP: Language-Script Aware Multilingual Pretraining
by: Liu, Yihong, et al.
Published: (2024)

UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
by: Wang, Haoyu, et al.
Published: (2024)

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis
by: Wang, Mingyang, et al.
Published: (2023)

MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching
by: Schmidt, Fabian David, et al.
Published: (2025)

Multilingual Language Model Pretraining using Machine-translated Data
by: Wang, Jiayi, et al.
Published: (2025)

Factual Consistency of Multilingual Pretrained Language Models
by: Fierro, Constanza, et al.
Published: (2022)

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives
by: Li, Zihao, et al.
Published: (2024)

Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders
by: Mazumder, Debajyoti, et al.
Published: (2026)

Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining
by: Körner, Felicia, et al.
Published: (2026)

ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs
by: Yoo, HanGyeol, et al.
Published: (2026)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
by: Longpre, Shayne, et al.
Published: (2025)

CUTE: A Multilingual Dataset for Enhancing Cross-Lingual Knowledge Transfer in Low-Resource Languages
by: Zhuang, Wenhao, et al.
Published: (2025)

DA-Cramming: Enhancing Cost-Effective Language Model Pretraining with Dependency Agreement Integration
by: Kuo, Martin, et al.
Published: (2023)

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints
by: Song, Ran, et al.
Published: (2024)

PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data
by: Watts, Ishaan, et al.
Published: (2024)

Mix-of-Language-Experts Architecture for Multilingual Programming
by: Zong, Yifan, et al.
Published: (2025)

Datasets for Multilingual Answer Sentence Selection
by: Gabburo, Matteo, et al.
Published: (2024)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection
by: Messmer, Bettina, et al.
Published: (2025)

How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models?
by: Liu, Danni, et al.
Published: (2023)

GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text
by: Ginn, Michael, et al.
Published: (2024)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
by: Foroutan, Negar, et al.
Published: (2025)

Pretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages
by: Zhao, Yue, et al.
Published: (2026)