Multilingual Pretraining for Pixel Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kesen, Ilker, Lotz, Jonas F., Ziegler, Ingo, Rust, Phillip, Elliott, Desmond
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Role of Data Curation in Image Captioning
by: Li, Wenyan, et al.
Published: (2023)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by: Ziegler, Ingo, et al.
Published: (2024)

Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
by: Er, Yakup Abrek, et al.
Published: (2025)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
by: Guo, Ping, et al.
Published: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
by: Foroutan, Negar, et al.
Published: (2025)

Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare
by: Acikgoz, Emre Can, et al.
Published: (2024)

CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
by: Zhang, Mike, et al.
Published: (2026)

How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models?
by: Liu, Danni, et al.
Published: (2023)

ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models
by: Villegas, Danae Sánchez, et al.
Published: (2025)

MOSAIC: A Multilingual, Taxonomy-Agnostic, and Computationally Efficient Approach for Radiological Report Classification
by: Schiavone, Alice, et al.
Published: (2025)

Can Community Notes Replace Professional Fact-Checkers?
by: Borenstein, Nadav, et al.
Published: (2025)

Vision-Language Models under Cultural and Inclusive Considerations
by: Karamolegkou, Antonia, et al.
Published: (2024)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
by: Ali, Mehdi, et al.
Published: (2025)

A-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation
by: Wang, Kesen, et al.
Published: (2025)

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
by: Villegas, Danae Sánchez, et al.
Published: (2026)

Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection
by: Turki, Yassine, et al.
Published: (2026)

Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
by: Su, Zeli, et al.
Published: (2025)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
by: Chen, Zhixun, et al.
Published: (2025)

Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
by: Mathewson, Kyle Elliott
Published: (2026)

Towards Privacy-Aware Sign Language Translation at Scale
by: Rust, Phillip, et al.
Published: (2024)

Accelerating Multilingual Language Model for Excessively Tokenized Languages
by: Hong, Jimin, et al.
Published: (2024)

Evaluating Spatial Understanding of Large Language Models
by: Yamada, Yutaro, et al.
Published: (2023)

On Linear Representations and Pretraining Data Frequency in Language Models
by: Merullo, Jack, et al.
Published: (2025)

Drop Dropout on Single-Epoch Language Model Pretraining
by: Liu, Houjun, et al.
Published: (2025)

Gender Encoding Patterns in Pretrained Language Model Representations
by: Zakizadeh, Mahdi, et al.
Published: (2025)

Rethinking the Role of Text Complexity in Language Model Pretraining
by: Velasco, Dan John, et al.
Published: (2025)

Beyond Binary Classification: Detecting Fine-Grained Sexism in Social Media Videos
by: De Grazia, Laura, et al.
Published: (2026)

Multilingual Collaborative Defense for Large Language Models
by: Li, Hongliang, et al.
Published: (2025)

Cross-lingual Editing in Multilingual Language Models
by: Beniwal, Himanshu, et al.
Published: (2024)

The Roles of English in Evaluating Multilingual Language Models
by: Poelman, Wessel, et al.
Published: (2024)

SUTRA: Scalable Multilingual Language Model Architecture
by: Bendale, Abhijit, et al.
Published: (2024)

Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
by: McLeish, Sean, et al.
Published: (2025)

Breaking Language Barriers: Equitable Performance in Multilingual Language Models
by: Nagar, Tanay, et al.
Published: (2025)

All Languages Matter: On the Multilingual Safety of Large Language Models
by: Wang, Wenxuan, et al.
Published: (2023)

EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training
by: Dorkin, Aleksei, et al.
Published: (2026)

Classification of Radiological Text in Small and Imbalanced Datasets in a Non-English Language
by: Beliveau, Vincent, et al.
Published: (2024)

Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models?
by: Chen, Pinzhen, et al.
Published: (2024)

PonderLM: Pretraining Language Models to Ponder in Continuous Space
by: Zeng, Boyi, et al.
Published: (2025)

Code Pretraining Improves Entity Tracking Abilities of Language Models
by: Kim, Najoung, et al.
Published: (2024)

Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation
by: Elhady, Ahmed, et al.
Published: (2025)