Cover Image

Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dobler, Konstantin, de Melo, Gerard
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Token Distillation: Attention-aware Input Embeddings For New Tokens
by: Dobler, Konstantin, et al.
Published: (2025)

I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
by: Cohen, Roi, et al.
Published: (2024)

FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
by: Dobler, Konstantin, et al.
Published: (2023)

Token-Budget-Aware LLM Reasoning
by: Han, Tingxu, et al.
Published: (2024)

Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation
by: Tang, Pingzhi, et al.
Published: (2026)

Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP
by: Remy, François, et al.
Published: (2024)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
by: Hagemann, Johannes, et al.
Published: (2023)

I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
by: Liu, Yuhang, et al.
Published: (2025)

HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
by: Chen, Lixian, et al.
Published: (2026)

LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
by: Zhou, Enshuai, et al.
Published: (2026)

Enhancing Latent Computation in Transformers with Latent Tokens
by: Sun, Yuchang, et al.
Published: (2025)

PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
by: Goffinet, Etienne, et al.
Published: (2025)

Screening Is Enough
by: Nakanishi, Ken M.
Published: (2026)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
by: Miyamoto, Sora, et al.
Published: (2026)

Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers
by: Yang, Wang, et al.
Published: (2026)

CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
by: Mohtashami, Amirkeivan, et al.
Published: (2023)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
by: Bandarkar, Lucas, et al.
Published: (2024)

Language Modeling with Learned Meta-Tokens
by: Shah, Alok N., et al.
Published: (2025)

Parallel Token Prediction for Language Models
by: Draxler, Felix, et al.
Published: (2025)

Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage
by: Dubey, Prasanjit, et al.
Published: (2026)

Stopping Computation for Converged Tokens in Masked Diffusion-LM Decoding
by: Oba, Daisuke, et al.
Published: (2026)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
by: Li, Shiwei, et al.
Published: (2025)

From Construction to Injection: Edit-Based Fingerprints for Large Language Models
by: Li, Yue, et al.
Published: (2025)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Capability-Guided Compression: Toward Interpretability-Aware Budget Allocation for Large Language Models
by: Gupta, Rishaank
Published: (2026)

From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction
by: Zhu, Mingcheng, et al.
Published: (2026)

Mitigate the Gap: Investigating Approaches for Improving Cross-Modal Alignment in CLIP
by: Eslami, Sedigheh, et al.
Published: (2024)

Small LLMs with Expert Blocks Are Good Enough for Hyperparamter Tuning
by: Naphade, Om, et al.
Published: (2025)

Induction Signatures Are Not Enough: A Matched-Compute Study of Load-Bearing Structure in In-Context Learning
by: Sabry, Mohammed, et al.
Published: (2025)

Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference
by: Sabry, Mohammed, et al.
Published: (2026)

The Foundations of Tokenization: Statistical and Computational Concerns
by: Gastaldi, Juan Luis, et al.
Published: (2024)

The Geometry of Tokens in Internal Representations of Large Language Models
by: Viswanathan, Karthik, et al.
Published: (2025)

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
by: Bae, Sangmin, et al.
Published: (2025)

API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access
by: Su, Jiayuan, et al.
Published: (2024)

Towards Linguistically-Aware and Language-Independent Tokenization for Large Language Models (LLMs)
by: Rahman, Abrar, et al.
Published: (2024)

Online Adaptation of Language Models with a Memory of Amortized Contexts
by: Tack, Jihoon, et al.
Published: (2024)

Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models
by: Li, Yue, et al.
Published: (2025)

Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training
by: Tran, Toan, et al.
Published: (2025)

Segmentation and Processing of German Court Decisions from Open Legal Data
by: Darji, Harshil, et al.
Published: (2026)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
by: Wu, Zihao, et al.
Published: (2026)