Cover Image

LBPE: Long-token-first Tokenization to Improve Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lian, Haoran, Xiong, Yizhe, Lin, Zijia, Niu, Jianwei, Mo, Shasha, Chen, Hui, Liu, Peng, Ding, Guiguang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
by: Lian, Haoran, et al.
Published: (2024)

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
by: Lian, Haoran, et al.
Published: (2024)

Temporal Scaling Law for Large Language Models
by: Xiong, Yizhe, et al.
Published: (2024)

Fast Quiet-STaR: Thinking Without Thought Tokens
by: Huang, Wei, et al.
Published: (2025)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
by: Xiong, Yizhe, et al.
Published: (2025)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
by: Su, Zhenpeng, et al.
Published: (2023)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
by: Pan, Leiyu, et al.
Published: (2025)

Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation
by: Xiong, Yizhe, et al.
Published: (2025)

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
by: Liang, Yiwen, et al.
Published: (2025)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
by: Lv, Minxuan, et al.
Published: (2025)

LSNet: See Large, Focus Small
by: Wang, Ao, et al.
Published: (2025)

[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
by: Wang, Ao, et al.
Published: (2024)

Why do LLMs attend to the first token?
by: Barbero, Federico, et al.
Published: (2025)

Better & Faster Large Language Models via Multi-token Prediction
by: Gloeckle, Fabian, et al.
Published: (2024)

Problematic Tokens: Tokenizer Bias in Large Language Models
by: Yang, Jin, et al.
Published: (2024)

Large Language Models Can Self-Improve in Long-context Reasoning
by: Li, Siheng, et al.
Published: (2024)

Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models
by: Yao, Jui-Ming, et al.
Published: (2025)

LongReward: Improving Long-context Large Language Models with AI Feedback
by: Zhang, Jiajie, et al.
Published: (2024)

AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
by: Sun, Fengyuan, et al.
Published: (2025)

PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
by: Xiong, Yizhe, et al.
Published: (2024)

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
by: Gao, Chaochen, et al.
Published: (2025)

RepViT-SAM: Towards Real-Time Segmenting Anything
by: Wang, Ao, et al.
Published: (2023)

RepViT: Revisiting Mobile CNN From ViT Perspective
by: Wang, Ao, et al.
Published: (2023)

MBTSAD: Mitigating Backdoors in Language Models Based on Token Splitting and Attention Distillation
by: Ding, Yidong, et al.
Published: (2025)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
by: Xiong, Chen, et al.
Published: (2025)

Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
by: Singh, Aaditya K., et al.
Published: (2024)

Beyond Token-Level Policy Gradients for Complex Reasoning with Large Language Models
by: Xu, Mufan, et al.
Published: (2026)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
by: Kwek, Eugene, et al.
Published: (2025)

Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability
by: Hao, Tianxiang, et al.
Published: (2023)

Large Language Model as Token Compressor and Decompressor
by: Li, Wenbing, et al.
Published: (2026)

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
by: Wang, Dixuan, et al.
Published: (2024)

HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability
by: Chen, Yanfang, et al.
Published: (2024)

Rethinking Personalization in Large Language Models at the Token Level
by: Zhang, Chenheng, et al.
Published: (2026)

Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models
by: Liu, Zefang, et al.
Published: (2025)

Improving Data and Reward Design for Scientific Reasoning in Large Language Models
by: Chen, Zijie, et al.
Published: (2026)

Tokenization Falling Short: On Subword Robustness in Large Language Models
by: Chai, Yekun, et al.
Published: (2024)

S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models
by: He, Tao, et al.
Published: (2025)

Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representations
by: Wang, Haoran, et al.
Published: (2026)