Cover Image

Diversity or Precision? A Deep Dive into Next Token Prediction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Haoyuan, Wang, Hai, Wu, Jiajia, Ou, Jinxiang, Wang, Keyao, Chen, Weile, Zheng, Zihao, Yu, Bei
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

One-Token Rollout: Guiding Supervised Fine-Tuning of LLMs with Policy Gradient
by: Ming, Rui, et al.
Published: (2025)

Divergent Thoughts toward One Goal: LLM-based Multi-Agent Collaboration System for Electronic Design Automation
by: Wu, Haoyuan, et al.
Published: (2025)

Efficient OpAmp Adaptation for Zoom Attention to Golden Contexts
by: Wu, Haoyuan, et al.
Published: (2025)

Cautious Next Token Prediction
by: Wang, Yizhou, et al.
Published: (2025)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
by: Wu, Zihao, et al.
Published: (2026)

NITP: Next Implicit Token Prediction for LLM Pre-training
by: Zhang, Xiangdong, et al.
Published: (2026)

Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension
by: Qi, Mengnan, et al.
Published: (2024)

Customized Retrieval Augmented Generation and Benchmarking for EDA Tool Documentation QA
by: Pu, Yuan, et al.
Published: (2024)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
by: Qing, Liu, et al.
Published: (2026)

ToTRL: Unlock LLM Tree-of-Thoughts Reasoning Potential through Puzzles Solving
by: Wu, Haoyuan, et al.
Published: (2025)

Next Token Perception Score: Analytical Assessment of your LLM Perception Skills
by: Cheng, Yu-Ang, et al.
Published: (2025)

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
by: Zhang, Jiebin, et al.
Published: (2024)

Reasoning Bias of Next Token Prediction Training
by: Lin, Pengxiao, et al.
Published: (2025)

ENTP: Encoder-only Next Token Prediction
by: Ewer, Ethan, et al.
Published: (2024)

NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction
by: Wang, Qichao, et al.
Published: (2025)

On-Policy Optimization with Group Equivalent Preference for Multi-Programming Language Understanding
by: Wu, Haoyuan, et al.
Published: (2025)

Unlearning What Matters: Token-Level Attribution for Precise Language Model Unlearning
by: Wu, Jiawei, et al.
Published: (2026)

Training LLMs Beyond Next Token Prediction -- Filling the Mutual Information Gap
by: Yang, Chun-Hao, et al.
Published: (2025)

Alternatives To Next Token Prediction In Text Generation -- A Survey
by: Wyatt, Charlie, et al.
Published: (2025)

YNTP-100: A Benchmark for Your Next Token Prediction with 100 People
by: Ding, Shiyao, et al.
Published: (2025)

From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
by: Tian, Yuchuan, et al.
Published: (2025)

Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference
by: Wu, Zimeng, et al.
Published: (2026)

Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
by: Mao, Yu, et al.
Published: (2025)

LongCat-Next: Lexicalizing Modalities as Discrete Tokens
by: Meituan LongCat Team, et al.
Published: (2026)

Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction
by: Qian, Junlang, et al.
Published: (2025)

Fractal Patterns May Illuminate the Success of Next-Token Prediction
by: Alabdulmohsin, Ibrahim, et al.
Published: (2024)

Implicit Optimization Bias of Next-Token Prediction in Linear Models
by: Thrampoulidis, Christos
Published: (2024)

Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
by: Chen, Liang, et al.
Published: (2024)

Reinforced Fast Weights with Next-Sequence Prediction
by: Hwang, Hee Seung, et al.
Published: (2026)

A Law of Next-Token Prediction in Large Language Models
by: He, Hangfeng, et al.
Published: (2024)

VocalNet-M2: Advancing Low-Latency Spoken Language Modeling via Integrated Multi-Codebook Tokenization and Multi-Token Prediction
by: Wang, Yuhao, et al.
Published: (2025)

Modeling Next-Token Prediction as Left-Nested Intuitionistic Implication
by: Tarau, Paul
Published: (2026)

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
by: Han, Minglun, et al.
Published: (2024)

On Support Samples of Next Word Prediction
by: Li, Yuqian, et al.
Published: (2025)

A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques
by: Thakkar, Megh, et al.
Published: (2024)

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm
by: Liu, Hongcheng, et al.
Published: (2024)

Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations
by: Zhao, Yize, et al.
Published: (2025)

NDP: Next Distribution Prediction as a More Broad Target
by: Ruan, Junhao, et al.
Published: (2024)

Mechanics of Next Token Prediction with Self-Attention
by: Li, Yingcong, et al.
Published: (2024)

Improving Next Tokens via Second-to-Last Predictions with Generate and Refine
by: Schneider, Johannes
Published: (2024)