Copertina

One Token Is Enough: Improving Diffusion Language Models with a Sink Token

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Zihou, Xie, Zheyong, Zhong, Li, Liu, Haifeng, Hu, Yao, Cao, Shaosheng
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training
di: Li, Shengrui, et al.
Pubblicazione: (2026)

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
di: Yao, Lin
Pubblicazione: (2026)

Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models
di: Zhong, Linhao, et al.
Pubblicazione: (2026)

Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models
di: Yao, Jui-Ming, et al.
Pubblicazione: (2025)

Counting Ability of Large Language Models and Impact of Tokenization
di: Zhang, Xiang, et al.
Pubblicazione: (2024)

OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
di: Shin, Seungjun, et al.
Pubblicazione: (2025)

IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
di: Liu, Ruikang, et al.
Pubblicazione: (2024)

Attention Sinks in Diffusion Language Models
di: Rulli, Maximo Eduardo, et al.
Pubblicazione: (2025)

Token-level Direct Preference Optimization
di: Zeng, Yongcheng, et al.
Pubblicazione: (2024)

Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
di: Liu, Peijie, et al.
Pubblicazione: (2025)

Saliency-driven Dynamic Token Pruning for Large Language Models
di: Tao, Yao, et al.
Pubblicazione: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
di: Liu, Dong, et al.
Pubblicazione: (2025)

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
di: Wang, Dixuan, et al.
Pubblicazione: (2024)

Full-ECE: A Metric For Token-level Calibration on Large Language Models
di: Liu, Han, et al.
Pubblicazione: (2024)

Thinking Tokens for Language Modeling
di: Herel, David, et al.
Pubblicazione: (2024)

Sink-Aware Pruning for Diffusion Language Models
di: Myrzakhan, Aidar, et al.
Pubblicazione: (2026)

KV Prediction for Improved Time to First Token
di: Horton, Maxwell, et al.
Pubblicazione: (2024)

LLM-Oriented Token-Adaptive Knowledge Distillation
di: Xie, Xurong, et al.
Pubblicazione: (2025)

Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models
di: Wu, Jialiang, et al.
Pubblicazione: (2025)

Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
di: Su, DiJia, et al.
Pubblicazione: (2025)

Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
di: Tian, Runchu, et al.
Pubblicazione: (2025)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
di: Cao, Qingqing, et al.
Pubblicazione: (2023)

The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
di: Sun, Bohang, et al.
Pubblicazione: (2026)

Re-Initialization Token Learning for Tool-Augmented Large Language Models
di: Li, Chenghao, et al.
Pubblicazione: (2025)

An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
di: Chai, Ziwei, et al.
Pubblicazione: (2024)

Memory Retrieval and Consolidation in Large Language Models through Function Tokens
di: Zhang, Shaohua, et al.
Pubblicazione: (2025)

Tokenization Matters: Improving Zero-Shot NER for Indic Languages
di: Pattnayak, Priyaranjan, et al.
Pubblicazione: (2025)

STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models
di: Sun, Xinhao, et al.
Pubblicazione: (2025)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
di: Cho, Gyeongje, et al.
Pubblicazione: (2025)

Accelerating Multilingual Language Model for Excessively Tokenized Languages
di: Hong, Jimin, et al.
Pubblicazione: (2024)

Probabilistic Token Alignment for Large Language Model Fusion
di: Zeng, Runjia, et al.
Pubblicazione: (2025)

GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
di: Hu, Shijing, et al.
Pubblicazione: (2025)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
di: Hu, Xiang, et al.
Pubblicazione: (2025)

The Token Tax: Systematic Bias in Multilingual Tokenization
di: Lundin, Jessica M., et al.
Pubblicazione: (2025)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
di: Kalamkar, Prathamesh, et al.
Pubblicazione: (2025)

Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
di: Zhang, Zhongjian, et al.
Pubblicazione: (2026)

Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
di: Kim, Woojin, et al.
Pubblicazione: (2025)

Token-Level Uncertainty-Aware Objective for Language Model Post-Training
di: Liu, Tingkai, et al.
Pubblicazione: (2025)

Balancing Understanding and Generation in Discrete Diffusion Models
di: Liu, Yue, et al.
Pubblicazione: (2026)

From Language Models over Tokens to Language Models over Characters
di: Vieira, Tim, et al.
Pubblicazione: (2024)