Cover Image

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kallini, Julie, Murty, Shikhar, Manning, Christopher D., Potts, Christopher, Csordás, Róbert
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space
by: Liu, Houjun, et al.
Published: (2025)

Do Language Models Use Their Depth Efficiently?
by: Csordás, Róbert, et al.
Published: (2025)

Mission: Impossible Language Models
by: Kallini, Julie, et al.
Published: (2024)

Fast Byte Latent Transformer
by: Kallini, Julie, et al.
Published: (2026)

Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations
by: Csordás, Róbert, et al.
Published: (2024)

MoEUT: Mixture-of-Experts Universal Transformers
by: Csordás, Róbert, et al.
Published: (2024)

Language models as tools for investigating the distinction between possible and impossible natural languages
by: Kallini, Julie, et al.
Published: (2025)

Sneaking Syntax into Transformer Language Models with Tree Regularization
by: Nandi, Ananjan, et al.
Published: (2024)

ReFT: Representation Finetuning for Language Models
by: Wu, Zhengxuan, et al.
Published: (2024)

SpaceByte: Towards Deleting Tokenization from Large Language Modeling
by: Slagle, Kevin
Published: (2024)

False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models
by: Kallini, Julie, et al.
Published: (2025)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors
by: Huang, Jing, et al.
Published: (2025)

Drop Dropout on Single-Epoch Language Model Pretraining
by: Liu, Houjun, et al.
Published: (2025)

NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild
by: Murty, Shikhar, et al.
Published: (2024)

Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluations
by: Agarwal, Ananth, et al.
Published: (2025)

BAGEL: Bootstrapping Agents by Guiding Exploration with Language
by: Murty, Shikhar, et al.
Published: (2024)

Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings
by: Gopalakrishnan, Anand, et al.
Published: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

Mechanistic evaluation of Transformers and state space models
by: Arora, Aryaman, et al.
Published: (2025)

Hypertokens: Holographic Associative Memory in Tokenized LLMs
by: Augeri, Christopher James
Published: (2025)

Fisher Mask Nodes for Language Model Merging
by: K, Thennal D, et al.
Published: (2024)

Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
by: Soylu, Dilara, et al.
Published: (2024)

AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
by: Wu, Zhengxuan, et al.
Published: (2025)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
by: Schröder, Christopher, et al.
Published: (2024)

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization
by: Foroutan, Negar, et al.
Published: (2025)

Dynamic Model Merging Made Slim
by: Du, Guodong, et al.
Published: (2026)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
by: Goddard, Charles, et al.
Published: (2024)

Token-Efficient Leverage Learning in Large Language Models
by: Zeng, Yuanhao, et al.
Published: (2024)

Language Model Cascades: Token-level uncertainty and beyond
by: Gupta, Neha, et al.
Published: (2024)

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
by: Toukmaji, Christopher
Published: (2024)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
by: Opsahl-Ong, Krista, et al.
Published: (2024)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
by: Glocker, Kevin, et al.
Published: (2025)

Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use
by: Goldie, Anna, et al.
Published: (2025)

Dynamic Thinking-Token Selection for Efficient Reasoning in Large Reasoning Models
by: Guo, Zhenyuan, et al.
Published: (2026)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
by: Cao, Qi, et al.
Published: (2026)

Language Models with Conformal Factuality Guarantees
by: Mohri, Christopher, et al.
Published: (2024)