Cover Image

One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ebing, Benedikt, Keller, Lennart, Glavaš, Goran
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages
by: Ebing, Benedikt, et al.
Published: (2023)

TransAlign: Machine Translation Encoders are Strong Word Aligners, Too
by: Ebing, Benedikt, et al.
Published: (2025)

The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Tasks
by: Ebing, Benedikt, et al.
Published: (2025)

SpeechTaxi: On Multilingual Semantic Speech Classification
by: Keller, Lennart, et al.
Published: (2024)

Geographic Adaptation of Pretrained Language Models
by: Hofmann, Valentin, et al.
Published: (2022)

Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
by: Huang, Yongxin, et al.
Published: (2024)

IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators
by: Paul, Indraneil, et al.
Published: (2024)

Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?
by: Geigle, Gregor, et al.
Published: (2024)

African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification
by: Geigle, Gregor, et al.
Published: (2024)

Romanized to Native Malayalam Script Transliteration Using an Encoder-Decoder Framework
by: Baiju, Bajiyo, et al.
Published: (2024)

Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations
by: Geigle, Gregor, et al.
Published: (2023)

Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages
by: Schmidt, Fabian David, et al.
Published: (2024)

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget
by: Bui, Minh Duc, et al.
Published: (2024)

Script Sensitivity: Benchmarking Language Models on Unicode, Romanized and Mixed-Script Sinhala
by: Rajapakse, Minuri, et al.
Published: (2026)

Supervised In-Context Fine-Tuning for Generative Sequence Labeling
by: Dukić, David, et al.
Published: (2025)

ReCoVeR the Target Language: Language Steering without Sacrificing Task Performance
by: Sterz, Hannah, et al.
Published: (2025)

MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching
by: Schmidt, Fabian David, et al.
Published: (2025)

ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection
by: Herrlein, Janek, et al.
Published: (2024)

How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination
by: Islam, Saad Obaid ul, et al.
Published: (2025)

Large Language Models are Miscalibrated In-Context Learners
by: Li, Chengzu, et al.
Published: (2023)

Compositional Steering of Large Language Models with Steering Tokens
by: Radevski, Gorjan, et al.
Published: (2026)

LangSAMP: Language-Script Aware Multilingual Pretraining
by: Liu, Yihong, et al.
Published: (2024)

SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU
by: Razumovskaia, Evgeniia, et al.
Published: (2023)

Should We Still Pretrain Encoders with Masked Language Modeling?
by: Gisserot-Boukhlef, Hippolyte, et al.
Published: (2025)

MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts
by: Hu, Chen, et al.
Published: (2026)

Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
by: Schmidt, Fabian David, et al.
Published: (2025)

One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations
by: Karne, Sripad
Published: (2026)

mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs
by: Geigle, Gregor, et al.
Published: (2023)

The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers
by: Islam, Saad Obaid ul, et al.
Published: (2025)

Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection
by: Dukić, David, et al.
Published: (2023)

Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
by: Geigle, Gregor, et al.
Published: (2025)

Script Gap: Evaluating LLM Triage on Indian Languages in Native vs Romanized Scripts in a Real World Setting
by: Khullar, Manurag, et al.
Published: (2025)

Don't Stop the Multi-Party! On Generating Synthetic Written Multi-Party Conversations with Constraints
by: Penzo, Nicolò, et al.
Published: (2025)

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models
by: Liu, Yihong, et al.
Published: (2024)

Select and Summarize: Scene Saliency for Movie Script Summarization
by: Saxena, Rohit, et al.
Published: (2024)

A Causal Language Modeling Detour Improves Encoder Continued Pretraining
by: Touchent, Rian, et al.
Published: (2026)

Pretraining and Benchmarking Modern Encoders for Latvian
by: Znotins, Arturs
Published: (2026)

Evaluating the Ability of LLMs to Solve Semantics-Aware Process Mining Tasks
by: Rebmann, Adrian, et al.
Published: (2024)

Renaissance: Investigating the Pretraining of Vision-Language Encoders
by: Fields, Clayton, et al.
Published: (2024)

Language Models on a Diet: Cost-Efficient Development of Encoders for Closely-Related Languages via Additional Pretraining
by: Ljubešić, Nikola, et al.
Published: (2024)