Image de couverture de livre

Efficient Pre-Training with Token Superposition

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Peng, Bowen, Gigant, Théo, Quesnelle, Jeffrey
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation
par: Gigant, Théo, et autres
Publié: (2026)

Long Context Pre-Training with Lighthouse Attention
par: Peng, Bowen, et autres
Publié: (2026)

YaRN: Efficient Context Window Extension of Large Language Models
par: Peng, Bowen, et autres
Publié: (2023)

Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure
par: Gigant, Théo, et autres
Publié: (2025)

Hermes 3 Technical Report
par: Teknium, Ryan, et autres
Publié: (2024)

Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metrics
par: Gigant, Théo, et autres
Publié: (2024)

Distilling Token-Trained Models into Byte-Level Models
par: Bao, Zishuo, et autres
Publié: (2026)

Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training
par: Chen, Yangyi, et autres
Publié: (2025)

Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pre-trained Models
par: Purason, Taido, et autres
Publié: (2025)

Gold-Switch: Training-Free Superposition of Slow- and Fast- Thinking LLMs
par: Lee, Jaeseong, et autres
Publié: (2025)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
par: Guo, Xu, et autres
Publié: (2026)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
par: Aynetdinov, Ansar, et autres
Publié: (2025)

From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMs
par: Nishu, Kumari, et autres
Publié: (2025)

Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model
par: Yuan, Chenhan, et autres
Publié: (2024)

Forest Before Trees: Latent Superposition for Efficient Visual Reasoning
par: Wang, Yubo, et autres
Publié: (2026)

Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
par: Lu, Keming, et autres
Publié: (2024)

SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings
par: SadraeiJavaeri, MohammadAli, et autres
Publié: (2024)

BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training
par: Chizhov, Pavel, et autres
Publié: (2024)

LLM Latent Reasoning as Chain of Superposition
par: Deng, Jingcheng, et autres
Publié: (2025)

Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing
par: Goel, Raghavv, et autres
Publié: (2026)

Adaptive Group Policy Optimization: Towards Stable Training and Token-Efficient Reasoning
par: Li, Chen, et autres
Publié: (2025)

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
par: Yu, Le, et autres
Publié: (2024)

Medical Vision-Language Pre-Training for Brain Abnormalities
par: Monajatipoor, Masoud, et autres
Publié: (2024)

Reinforcement Pre-Training
par: Dong, Qingxiu, et autres
Publié: (2025)

Entropy-Driven Pre-Tokenization for Byte-Pair Encoding
par: Hu, Yifan, et autres
Publié: (2025)

Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model
par: Ding, Bowen, et autres
Publié: (2025)

Efficient Training of Language Models with Compact and Consistent Next Token Distributions
par: Sathe, Ashutosh, et autres
Publié: (2024)

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
par: Han, Minglun, et autres
Publié: (2024)

On the Limits of Token Reduction for Efficient Unified Vision Language Training
par: Chen, Siyi, et autres
Publié: (2026)

NITP: Next Implicit Token Prediction for LLM Pre-training
par: Zhang, Xiangdong, et autres
Publié: (2026)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
par: Zhao, Bowen, et autres
Publié: (2024)

Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
par: Li, Wenhao, et autres
Publié: (2026)

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework
par: Zhang, Chenyuan, et autres
Publié: (2026)

Detecting Concrete Visual Tokens for Multimodal Machine Translation
par: Bowen, Braeden, et autres
Publié: (2024)

X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression
par: Li, Guihong, et autres
Publié: (2025)

Token Masking Improves Transformer-Based Text Classification
par: Xu, Xianglong, et autres
Publié: (2025)

Efficient Switchable Safety Control in LLMs via Magic-Token-Guided Co-Training
par: Si, Jianfeng, et autres
Publié: (2025)

Few-shot Named Entity Recognition via Superposition Concept Discrimination
par: Chen, Jiawei, et autres
Publié: (2024)

A General and Efficient Training for Transformer via Token Expansion
par: Huang, Wenxuan, et autres
Publié: (2024)

Beyond Tokens: Concept-Level Training Objectives for LLMs
par: Iyer, Laya, et autres
Publié: (2026)