Cover Image

Think before you speak: Training Language Models With Pause Tokens

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Goyal, Sachin, Ji, Ziwei, Rawat, Ankit Singh, Menon, Aditya Krishna, Kumar, Sanjiv, Nagarajan, Vaishnavh
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Language Model Cascades: Token-level uncertainty and beyond
by: Gupta, Neha, et al.
Published: (2024)

On student-teacher deviations in distillation: does it pay to disobey?
by: Nagarajan, Vaishnavh, et al.
Published: (2023)

Deep sequence models tend to memorize geometrically; it is unclear why
by: Noroozizadeh, Shahriar, et al.
Published: (2025)

Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
by: Nagarajan, Vaishnavh, et al.
Published: (2025)

Faster Cascades via Speculative Decoding
by: Narasimhan, Harikrishna, et al.
Published: (2024)

The pitfalls of next-token prediction
by: Bachmann, Gregor, et al.
Published: (2024)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Regression-aware Inference with LLMs
by: Lukasik, Michal, et al.
Published: (2024)

Cascade-Aware Training of Language Models
by: Wang, Congchao, et al.
Published: (2024)

Efficient Document Ranking with Learnable Late Interactions
by: Ji, Ziwei, et al.
Published: (2024)

Thinking Tokens for Language Modeling
by: Herel, David, et al.
Published: (2024)

A Statistical Framework for Data-dependent Retrieval-Augmented Models
by: Basu, Soumya, et al.
Published: (2024)

Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
by: Vakilian, Vala, et al.
Published: (2025)

Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
by: Singla, Pratham, et al.
Published: (2025)

Mechanics of Next Token Prediction with Self-Attention
by: Li, Yingcong, et al.
Published: (2024)

Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production
by: Galashov, Alexandre, et al.
Published: (2025)

Large Language Models aren't all that you need
by: Holla, Kiran Voderhobli, et al.
Published: (2024)

When Does Confidence-Based Cascade Deferral Suffice?
by: Jitkrittum, Wittawat, et al.
Published: (2023)

Universal Model Routing for Efficient LLM Inference
by: Jitkrittum, Wittawat, et al.
Published: (2025)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
by: Kumar, Sachin
Published: (2026)

Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond
by: Li, Yingcong, et al.
Published: (2024)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
by: Rawat, Ankit Singh, et al.
Published: (2024)

Pause or Fabricate? Training Language Models for Grounded Reasoning
by: Qiu, Yiwen, et al.
Published: (2026)

Overtrained Language Models Are Harder to Fine-Tune
by: Springer, Jacob Mitchell, et al.
Published: (2025)

Mind the Pause: Disfluency-Aware Objective Tuning for Multilingual Speech Correction with LLMs
by: Kumar, Deepak, et al.
Published: (2026)

Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers
by: Yang, Wang, et al.
Published: (2026)

From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
by: Ildiz, M. Emrullah, et al.
Published: (2024)

Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration
by: Begin, James, et al.
Published: (2025)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
by: Shen, Junzhe, et al.
Published: (2026)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
by: Aynetdinov, Ansar, et al.
Published: (2025)

Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models
by: Xu, Zihao, et al.
Published: (2026)

FLEXITOKENS: Flexible Tokenization for Evolving Language Models
by: Owodunni, Abraham Toluwase, et al.
Published: (2025)

Token-Level Uncertainty-Aware Objective for Language Model Post-Training
by: Liu, Tingkai, et al.
Published: (2025)

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers
by: Wen, Yuxin, et al.
Published: (2024)

Thinking Ahead: Prospection-Guided Retrieval of Memory with Language Models
by: Chopra, Harshita, et al.
Published: (2026)

Online Training of Large Language Models: Learn while chatting
by: Liang, Juhao, et al.
Published: (2024)

Mode-Conditioning Unlocks Superior Test-Time Scaling
by: Wu, Chen Henry, et al.
Published: (2025)

Reasoning Up the Instruction Ladder for Controllable Language Models
by: Zheng, Zishuo, et al.
Published: (2025)

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies
by: Ovalle, Anaelia, et al.
Published: (2023)

Cross-Domain Content Generation with Domain-Specific Small Language Models
by: Maloo, Ankit, et al.
Published: (2024)