Image de couverture de livre

One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Abagyan, Diana, Salamanca, Alejandro R., Cruz-Salinas, Andres Felipe, Cao, Kris, Lin, Hangyu, Locatelli, Acyr, Fadaee, Marzieh, Üstün, Ahmet, Hooker, Sara
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
par: Gritsch, Nikolas, et autres
Publié: (2024)

To Code, or Not To Code? Exploring Impact of Code in Pre-training
par: Aryabumi, Viraat, et autres
Publié: (2024)

A Post-trainer's Guide to Multilingual Training Data: Uncovering Cross-lingual Transfer Dynamics
par: Shimabucoro, Luisa, et autres
Publié: (2025)

Tiny Aya: Bridging Scale and Multilingual Depth
par: Salamanca, Alejandro R., et autres
Publié: (2026)

Efficient Benchmarking Is Just Feature Selection and Multiple Regression
par: Bowyer, Sam, et autres
Publié: (2026)

Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
par: Ahmadian, Arash, et autres
Publié: (2024)

Aya 23: Open Weight Releases to Further Multilingual Progress
par: Aryabumi, Viraat, et autres
Publié: (2024)

The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
par: Mora, David, et autres
Publié: (2025)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
par: Aakanksha, et autres
Publié: (2024)

RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
par: Dang, John, et autres
Publié: (2024)

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives
par: Shimabucoro, Luísa, et autres
Publié: (2024)

Automatic WordNet Construction Using Markov Chain Monte Carlo
par: Marzieh Fadaee
Publié: (2013)

Aya Vision: Advancing the Frontier of Multilingual Multimodality
par: Dash, Saurabh, et autres
Publié: (2025)

How Does Quantization Affect Multilingual LLMs?
par: Marchisio, Kelly, et autres
Publié: (2024)

SimMerge: Learning to Select Merge Operators from Similarity Signals
par: Bolton, Oliver, et autres
Publié: (2026)

One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos
par: Bai, Zechen, et autres
Publié: (2024)

The Multilingual Divide and Its Impact on Global AI Safety
par: Peppin, Aidan, et autres
Publié: (2025)

Verification Limits Code LLM Training
par: Gureja, Srishti, et autres
Publié: (2025)

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
par: Aakanksha, et autres
Publié: (2024)

The Disparate Impacts of Speculative Decoding
par: Sandler, Jameson, et autres
Publié: (2025)

TabPFN: One Model to Rule Them All?
par: Zhang, Qiong, et autres
Publié: (2025)

SupeRANSAC: One RANSAC to Rule Them All
par: Barath, Daniel
Publié: (2025)

Repr Types: One Abstraction to Rule Them All
par: Palmkvist, Viktor, et autres
Publié: (2024)

OmniCaptioner: One Captioner to Rule Them All
par: Lu, Yiting, et autres
Publié: (2025)

Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers
par: D'souza, Daniel, et autres
Publié: (2025)

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
par: Üstün, Ahmet, et autres
Publié: (2024)

One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them
par: Holmov, Ali, et autres
Publié: (2026)

M-RewardBench: Evaluating Reward Models in Multilingual Settings
par: Gureja, Srishti, et autres
Publié: (2024)

One Prompt To Rule Them All: LLMs for Opinion Summary Evaluation
par: Siledar, Tejpalsingh, et autres
Publié: (2024)

One Model to Rule them All … and Through Emergence, Bind Them
par: Jawahar Bhalla, et autres
Publié: (2024)

Rope to Nope and Back Again: A New Hybrid Attention Strategy
par: Yang, Bowen, et autres
Publié: (2025)

Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation
par: Kreutzer, Julia, et autres
Publié: (2025)

The Leaderboard Illusion
par: Singh, Shivalika, et autres
Publié: (2025)

Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
par: Dang, John, et autres
Publié: (2024)

NeoBabel: A Multilingual Open Tower for Visual Generation
par: Derakhshani, Mohammad Mahdi, et autres
Publié: (2025)

Diversify and Conquer: Diversity-Centric Data Selection with Iterative Refinement
par: Yu, Simon, et autres
Publié: (2024)

One Instrument to Rule Them All: The Bias and Coverage of Just-ID IV
par: Angrist, Joshua, et autres
Publié: (2021)

One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation
par: Wei, Zhenyu, et autres
Publié: (2026)

The Token Tax: Systematic Bias in Multilingual Tokenization
par: Lundin, Jessica M., et autres
Publié: (2025)

One Extension to Explain Them All, One Scale-Invariant Spectrum to Test Them All, and in One Model Bind Them
par: Forconi, Matteo, et autres
Publié: (2025)