Cover Image

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shin, Haebin, Ji, Lei, Liu, Xiao, Gong, Yeyun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Generative Prompt Internalization
by: Shin, Haebin, et al.
Published: (2024)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
by: Tao, Chaofan, et al.
Published: (2024)

Lossless Vocabulary Reduction for Auto-Regressive Language Models
by: Chijiwa, Daiki, et al.
Published: (2025)

DVAGen: Dynamic Vocabulary Augmented Generation
by: Du, Wei, et al.
Published: (2025)

BiasJailbreak:Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Models
by: Lee, Isack, et al.
Published: (2024)

Fast Vocabulary Transfer for Language Model Compression
by: Gee, Leonidas, et al.
Published: (2024)

Process-based Self-Rewarding Language Models
by: Zhang, Shimao, et al.
Published: (2025)

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
by: Liu, Dong, et al.
Published: (2025)

Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models
by: Kim, Seungduk, et al.
Published: (2024)

How Vocabulary Sharing Facilitates Multilingualism in LLaMA?
by: Yuan, Fei, et al.
Published: (2023)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
by: Li, Haoling, et al.
Published: (2024)

An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference
by: Yamaguchi, Atsuki, et al.
Published: (2024)

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
by: Katz, Shahar, et al.
Published: (2024)

Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation
by: Linder, Max Rehman
Published: (2025)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
by: Kalamkar, Prathamesh, et al.
Published: (2025)

Scaling LLM Pre-training with Vocabulary Curriculum
by: Yu, Fangyuan
Published: (2025)

VocabTailor: Dynamic Vocabulary Selection for Downstream Tasks in Small Language Models
by: Zhang, Hanling, et al.
Published: (2025)

Open-Vocabulary Object Detection via Language Hierarchy
by: Huang, Jiaxing, et al.
Published: (2024)

Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
by: Dineen, Jacob, et al.
Published: (2026)

How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective
by: Zhang, Shimao, et al.
Published: (2025)

Using Perspectival Words Is Harder Than Vocabulary Words for Humans and Even More So for Multimodal Language Models
by: Dong, Dota Tianai, et al.
Published: (2025)

A Language-agnostic Model of Child Language Acquisition
by: Mahon, Louis, et al.
Published: (2024)

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
by: Zhao, Weilin, et al.
Published: (2025)

DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models
by: Zhang, Jinbin, et al.
Published: (2025)

Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries
by: Sakajo, Haruki, et al.
Published: (2025)

We Can't Understand AI Using our Existing Vocabulary
by: Hewitt, John, et al.
Published: (2025)

Happiness is Sharing a Vocabulary: A Study of Transliteration Methods
by: Jung, Haeji, et al.
Published: (2025)

A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understanding
by: Safa, Abdulfattah, et al.
Published: (2024)

Column Vocabulary Association (CVA): semantic interpretation of dataless tables
by: Martorana, Margherita, et al.
Published: (2024)

Now It Sounds Like You: Learning Personalized Vocabulary On Device
by: Wang, Sid, et al.
Published: (2023)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
by: Yang, Kailai, et al.
Published: (2025)

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models
by: Sun, Jiashuo, et al.
Published: (2023)

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
by: Gou, Zhibin, et al.
Published: (2023)

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size
by: Hayou, Soufiane, et al.
Published: (2025)

How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?
by: Yamaguchi, Atsuki, et al.
Published: (2024)

Vocabulary embeddings organize linguistic structure early in language model training
by: Papadimitriou, Isabel, et al.
Published: (2025)

Dynamic Vocabulary Pruning in Early-Exit LLMs
by: Vincenti, Jort, et al.
Published: (2024)

Small Vocabularies, Big Gains: Pretraining and Tokenization in Time Series Models
by: Roger, Alexis, et al.
Published: (2025)

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
by: Ma, Ziqiao, et al.
Published: (2023)