Cover Image

Compute Optimal Tokenization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Limisiewicz, Tomasz, Pagnoni, Artidoro, Iyer, Srini, Lewis, Mike, Mehta, Sachin, Liu, Alisa, Li, Margaret, Ghosh, Gargi, Zettlemoyer, Luke
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Fast Byte Latent Transformer
by: Kallini, Julie, et al.
Published: (2026)

Byte Latent Transformer: Patches Scale Better Than Tokens
by: Pagnoni, Artidoro, et al.
Published: (2024)

Predicting vs. Acting: A Trade-off Between World Modeling & Agent Modeling
by: Li, Margaret, et al.
Published: (2024)

MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling
by: Limisiewicz, Tomasz, et al.
Published: (2024)

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
by: Blevins, Terra, et al.
Published: (2024)

Demystifying Prompts in Language Models via Perplexity Estimation
by: Gonen, Hila, et al.
Published: (2022)

Beyond Literal Token Overlap: Token Alignability for Multilinguality
by: Hämmerl, Katharina, et al.
Published: (2025)

MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization
by: Ahia, Orevaoghene, et al.
Published: (2024)

Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
by: Liang, Weixin, et al.
Published: (2024)

Slicing and Dicing: Configuring Optimal Mixtures of Experts
by: Li, Margaret, et al.
Published: (2026)

Latent Speech-Text Transformer
by: Lu, Yen-Ju, et al.
Published: (2025)

Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation
by: Limisiewicz, Tomasz, et al.
Published: (2025)

Bolmo: Byteifying the Next Generation of Language Models
by: Minixhofer, Benjamin, et al.
Published: (2025)

Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models
by: Gonen, Hila, et al.
Published: (2024)

Debiasing Algorithm through Model Adaptation
by: Limisiewicz, Tomasz, et al.
Published: (2023)

Memory Layers at Scale
by: Berges, Vincent-Pierre, et al.
Published: (2024)

Improving Factuality with Explicit Working Memory
by: Chen, Mingda, et al.
Published: (2024)

(Mis)Fitting: A Survey of Scaling Laws
by: Li, Margaret, et al.
Published: (2025)

Continual Learning via Sparse Memory Finetuning
by: Lin, Jessy, et al.
Published: (2025)

Self-Alignment with Instruction Backtranslation
by: Li, Xian, et al.
Published: (2023)

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
by: Liu, Jiacheng, et al.
Published: (2024)

Computational Tradeoffs in Image Synthesis: Diffusion, Masked-Token, and Next-Token Prediction
by: Kilian, Maciej, et al.
Published: (2024)

Demystifying CLIP Data
by: Xu, Hu, et al.
Published: (2023)

Teaching LLMs at Charles University: Assignments and Activities
by: Helcl, Jindřich, et al.
Published: (2024)

Comparing Hallucination Detection Metrics for Multilingual Generation
by: Kang, Haoqiang, et al.
Published: (2024)

Compute Optimal Scaling of Skills: Knowledge vs Reasoning
by: Roberts, Nicholas, et al.
Published: (2025)

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers
by: Wen, Yuxin, et al.
Published: (2024)

MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
by: Lin, Xi Victoria, et al.
Published: (2024)

KV Prediction for Improved Time to First Token
by: Horton, Maxwell, et al.
Published: (2024)

Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
by: Zheng, Brian Siyuan, et al.
Published: (2025)

Text Quality-Based Pruning for Efficient Training of Language Models
by: Sharma, Vasu, et al.
Published: (2024)

Are you going to finish that? A Practical Study of the Partial Token Problem
by: Xu, Hao, et al.
Published: (2026)

A Taxonomy of Ambiguity Types for NLP
by: Li, Margaret Y., et al.
Published: (2024)

In-context Pretraining: Language Modeling Beyond Document Boundaries
by: Shi, Weijia, et al.
Published: (2023)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
by: Fu, Qichen, et al.
Published: (2024)

From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMs
by: Nishu, Kumari, et al.
Published: (2025)

FLEXITOKENS: Flexible Tokenization for Evolving Language Models
by: Owodunni, Abraham Toluwase, et al.
Published: (2025)

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
by: Shao, Rulin, et al.
Published: (2024)

Altogether: Image Captioning via Re-aligning Alt-text
by: Xu, Hu, et al.
Published: (2024)

Micro Language Models Enable Instant Responses
by: Cheng, Wen, et al.
Published: (2026)