Image de couverture de livre

Distilling Token-Trained Models into Byte-Level Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bao, Zishuo, Leng, Jiaqi, Wang, Junxiong, Peng, Bowen, Lu, Yucheng
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation
par: Gigant, Théo, et autres
Publié: (2026)

MambaByte: Token-free Selective State Space Model
par: Wang, Junxiong, et autres
Publié: (2024)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
par: Leng, Jiaqi, et autres
Publié: (2025)

Cross-Tokenizer LLM Distillation through a Byte-Level Interface
par: Singh, Avyav Kumar, et autres
Publié: (2026)

Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation
par: Wei, Jingxuan, et autres
Publié: (2024)

Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers
par: Jang, Eugene, et autres
Publié: (2024)

Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles
par: Phan, Buu, et autres
Publié: (2024)

Efficient Pre-Training with Token Superposition
par: Peng, Bowen, et autres
Publié: (2026)

BanglaByT5: Byte-Level Modelling for Bangla
par: Bhattacharyya, Pramit, et autres
Publié: (2025)

ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
par: Deng, Chunyuan, et autres
Publié: (2026)

Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions
par: Bao, Zishuo, et autres
Publié: (2025)

Kathleen: Oscillator-Based Byte-Level Text Classification Without Tokenization or Attention
par: Fountzoulas, George
Publié: (2026)

GPUTOK: GPU Accelerated Byte Level BPE Tokenization
par: Kadamba, Venu Gopal, et autres
Publié: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
par: Zhang, Songming, et autres
Publié: (2025)

Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
par: Cui, Xiao, et autres
Publié: (2024)

Rethinking Personalization in Large Language Models at the Token Level
par: Zhang, Chenheng, et autres
Publié: (2026)

Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
par: Shravan, Rohan
Publié: (2026)

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
par: Lian, Haoran, et autres
Publié: (2024)

Byte BPE Tokenization as an Inverse string Homomorphism
par: Geng, Saibo, et autres
Publié: (2024)

Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via Markov Likelihood
par: Lin, Xingyu, et autres
Publié: (2025)

Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via Sequence-Level Likelihood
par: Lin, Xingyu, et autres
Publié: (2026)

Token-Level Uncertainty-Aware Objective for Language Model Post-Training
par: Liu, Tingkai, et autres
Publié: (2025)

Entropy-Driven Pre-Tokenization for Byte-Pair Encoding
par: Hu, Yifan, et autres
Publié: (2025)

Back to Bytes: Revisiting Tokenization Through UTF-8
par: Moryossef, Amit, et autres
Publié: (2025)

SpaceByte: Towards Deleting Tokenization from Large Language Modeling
par: Slagle, Kevin
Publié: (2024)

Hybrid Tokenization Strategy for DNA Language Model using Byte Pair Encoding and K-MER Methods
par: Sapkota, Ganesh, et autres
Publié: (2025)

EGAD: Entropy-Guided Adaptive Distillation for Token-Level Knowledge Transfer
par: Zhang, Hao, et autres
Publié: (2026)

CNMBERT: A Model for Converting Hanyu Pinyin Abbreviations to Chinese Characters
par: Feng, Zishuo, et autres
Publié: (2024)

Byte Latent Transformer: Patches Scale Better Than Tokens
par: Pagnoni, Artidoro, et autres
Publié: (2024)

Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model
par: Yuan, Chenhan, et autres
Publié: (2024)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
par: Kallini, Julie, et autres
Publié: (2024)

Beyond Tokens: Concept-Level Training Objectives for LLMs
par: Iyer, Laya, et autres
Publié: (2026)

Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
par: Wang, Jiapeng, et autres
Publié: (2025)

From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities
par: Zhang, Wanpeng, et autres
Publié: (2024)

Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling
par: Zhang, Zhen, et autres
Publié: (2026)

Discriminative Policy Optimization for Token-Level Reward Models
par: Chen, Hongzhan, et autres
Publié: (2025)

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models
par: Wang, Chengyu, et autres
Publié: (2025)

Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models
par: Zheng, Lin, et autres
Publié: (2026)

Reflection Pretraining Enables Token-Level Self-Correction in Biological Sequence Models
par: Zhang, Xiang, et autres
Publié: (2025)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
par: Shao, Chenze, et autres
Publié: (2024)