Image de couverture de livre

LBPE: Long-token-first Tokenization to Improve Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lian, Haoran, Xiong, Yizhe, Lin, Zijia, Niu, Jianwei, Mo, Shasha, Chen, Hui, Liu, Peng, Ding, Guiguang
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
par: Lian, Haoran, et autres
Publié: (2024)

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
par: Lian, Haoran, et autres
Publié: (2024)

Temporal Scaling Law for Large Language Models
par: Xiong, Yizhe, et autres
Publié: (2024)

Fast Quiet-STaR: Thinking Without Thought Tokens
par: Huang, Wei, et autres
Publié: (2025)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
par: Su, Zhenpeng, et autres
Publié: (2024)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
par: Xiong, Yizhe, et autres
Publié: (2025)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
par: Su, Zhenpeng, et autres
Publié: (2023)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
par: Su, Zhenpeng, et autres
Publié: (2024)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
par: Pan, Leiyu, et autres
Publié: (2025)

Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation
par: Xiong, Yizhe, et autres
Publié: (2025)

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
par: Liang, Yiwen, et autres
Publié: (2025)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
par: Lv, Minxuan, et autres
Publié: (2025)

LSNet: See Large, Focus Small
par: Wang, Ao, et autres
Publié: (2025)

[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
par: Wang, Ao, et autres
Publié: (2024)

Why do LLMs attend to the first token?
par: Barbero, Federico, et autres
Publié: (2025)

Better & Faster Large Language Models via Multi-token Prediction
par: Gloeckle, Fabian, et autres
Publié: (2024)

Problematic Tokens: Tokenizer Bias in Large Language Models
par: Yang, Jin, et autres
Publié: (2024)

Large Language Models Can Self-Improve in Long-context Reasoning
par: Li, Siheng, et autres
Publié: (2024)

Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models
par: Yao, Jui-Ming, et autres
Publié: (2025)

LongReward: Improving Long-context Large Language Models with AI Feedback
par: Zhang, Jiajie, et autres
Publié: (2024)

AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
par: Sun, Fengyuan, et autres
Publié: (2025)

PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
par: Xiong, Yizhe, et autres
Publié: (2024)

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
par: Gao, Chaochen, et autres
Publié: (2025)

MBTSAD: Mitigating Backdoors in Language Models Based on Token Splitting and Attention Distillation
par: Ding, Yidong, et autres
Publié: (2025)

RepViT-SAM: Towards Real-Time Segmenting Anything
par: Wang, Ao, et autres
Publié: (2023)

RepViT: Revisiting Mobile CNN From ViT Perspective
par: Wang, Ao, et autres
Publié: (2023)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
par: Xiong, Chen, et autres
Publié: (2025)

Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
par: Singh, Aaditya K., et autres
Publié: (2024)

Beyond Token-Level Policy Gradients for Complex Reasoning with Large Language Models
par: Xu, Mufan, et autres
Publié: (2026)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
par: Kwek, Eugene, et autres
Publié: (2025)

Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability
par: Hao, Tianxiang, et autres
Publié: (2023)

Large Language Model as Token Compressor and Decompressor
par: Li, Wenbing, et autres
Publié: (2026)

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
par: Wang, Dixuan, et autres
Publié: (2024)

HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability
par: Chen, Yanfang, et autres
Publié: (2024)

Rethinking Personalization in Large Language Models at the Token Level
par: Zhang, Chenheng, et autres
Publié: (2026)

Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models
par: Liu, Zefang, et autres
Publié: (2025)

Improving Data and Reward Design for Scientific Reasoning in Large Language Models
par: Chen, Zijie, et autres
Publié: (2026)

Tokenization Falling Short: On Subword Robustness in Large Language Models
par: Chai, Yekun, et autres
Publié: (2024)

S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models
par: He, Tao, et autres
Publié: (2025)

Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representations
par: Wang, Haoran, et autres
Publié: (2026)