Cover Image

Batching BPE Tokenization Merges

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Morgan, Alexander P.
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment
by: Vemula, Saketh Reddy, et al.
Published: (2025)

GPUTOK: GPU Accelerated Byte Level BPE Tokenization
by: Kadamba, Venu Gopal, et al.
Published: (2026)

MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies
by: Asgari, Ehsaneddin, et al.
Published: (2025)

LiteToken: Removing Intermediate Merge Residues From BPE Tokenizers
by: Sun, Yike, et al.
Published: (2026)

TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding
by: Wu, Zhaoxuan, et al.
Published: (2025)

Towards Nepali-language LLMs: Efficient GPT training with a Nepali BPE tokenizer
by: Shrestha, Adarsha, et al.
Published: (2025)

BlockBPE: Parallel BPE Tokenization
by: You, Amos
Published: (2025)

Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models
by: Xu, Zihao, et al.
Published: (2026)

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
by: Zheng, Zhen, et al.
Published: (2024)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Extrapolation Merging: Keep Improving With Extrapolation and Merging
by: Lin, Yiguan, et al.
Published: (2025)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
by: Shen, Yikang, et al.
Published: (2024)

SuperMerge: An Approach For Gradient-Based Model Merging
by: Yang, Haoyu, et al.
Published: (2024)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
by: Kallini, Julie, et al.
Published: (2024)

Transport and Merge: Cross-Architecture Merging for Large Language Models
by: Cui, Chenhang, et al.
Published: (2026)

MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs
by: Zhou, Yuhang, et al.
Published: (2025)

SE-Merging: A Self-Enhanced Approach for Dynamic Model Merging
by: Chen, Zijun, et al.
Published: (2025)

AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning
by: Zhong, Yiwu, et al.
Published: (2024)

Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Batch Speculative Decoding Done Right
by: Zhang, Ranran Haoran, et al.
Published: (2025)

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
by: Liu, Deyuan, et al.
Published: (2024)

LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models
by: Shang, Yuzhang, et al.
Published: (2024)

LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
by: Liu, Zehua, et al.
Published: (2025)

LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint
by: Ma, Qianli, et al.
Published: (2025)

AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging
by: Zhao, Yiran, et al.
Published: (2024)

RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior
by: Yang, Junyao, et al.
Published: (2025)

Channel Merging: Preserving Specialization for Merged Experts
by: Zhang, Mingyang, et al.
Published: (2024)

Reasoning Pattern Alignment Merging for Adaptive Reasoning
by: Zhong, Zhaofeng, et al.
Published: (2026)

Activation-Informed Merging of Large Language Models
by: Nobari, Amin Heyrani, et al.
Published: (2025)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

The Token Tax: Systematic Bias in Multilingual Tokenization
by: Lundin, Jessica M., et al.
Published: (2025)

Constructing a BPE Tokenization DFA
by: Berglund, Martin, et al.
Published: (2024)

Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Data
by: Hui, Charco, et al.
Published: (2025)

Unlocking the Potential of Model Merging for Low-Resource Languages
by: Tao, Mingxu, et al.
Published: (2024)

HM3: Heterogeneous Multi-Class Model Merging
by: Hackmann, Stefan
Published: (2024)

Merging Improves Self-Critique Against Jailbreak Attacks
by: Gallego, Victor
Published: (2024)

State over Tokens: Characterizing the Role of Reasoning Tokens
by: Levy, Mosh, et al.
Published: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
by: Wang, Zhixiang, et al.
Published: (2025)

Byte BPE Tokenization as an Inverse string Homomorphism
by: Geng, Saibo, et al.
Published: (2024)