Imagen de Portada

On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithms

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Trauger, Jacob, Tewari, Ambuj
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Characterizing the Multiclass Learnability of Forgiving 0-1 Loss Functions
por: Trauger, Jacob, et al.
Publicado: (2025)

Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
por: Huang, Vincent, et al.
Publicado: (2025)

Efficient Training of Language Models with Compact and Consistent Next Token Distributions
por: Sathe, Ashutosh, et al.
Publicado: (2024)

Reasoning Bias of Next Token Prediction Training
por: Lin, Pengxiao, et al.
Publicado: (2025)

ENTP: Encoder-only Next Token Prediction
por: Ewer, Ethan, et al.
Publicado: (2024)

A Characterization of List Language Identification in the Limit
por: Charikar, Moses, et al.
Publicado: (2025)

Cautious Next Token Prediction
por: Wang, Yizhou, et al.
Publicado: (2025)

Improving Self Consistency in LLMs through Probabilistic Tokenization
por: Sathe, Ashutosh, et al.
Publicado: (2024)

Implicit Optimization Bias of Next-Token Prediction in Linear Models
por: Thrampoulidis, Christos
Publicado: (2024)

Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension
por: Qi, Mengnan, et al.
Publicado: (2024)

A Training-free Method for LLM Text Attribution
por: Radvand, Tara, et al.
Publicado: (2025)

Improving Next Tokens via Second-to-Last Predictions with Generate and Refine
por: Schneider, Johannes
Publicado: (2024)

An Asymptotically Optimal Algorithm for the Convex Hull Membership Problem
por: Qiao, Gang, et al.
Publicado: (2023)

Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
por: Gong, Zixuan, et al.
Publicado: (2025)

Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
por: Mao, Yu, et al.
Publicado: (2025)

LatentQA: Teaching LLMs to Decode Activations Into Natural Language
por: Pan, Alexander, et al.
Publicado: (2024)

A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs
por: Hong, Kihyuk, et al.
Publicado: (2024)

If generative AI is the answer, what is the question?
por: Tewari, Ambuj
Publicado: (2025)

A Law of Next-Token Prediction in Large Language Models
por: He, Hangfeng, et al.
Publicado: (2024)

Differentially Private Next-Token Prediction of Large Language Models
por: Flemings, James, et al.
Publicado: (2024)

The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices
por: Arias, Esteban Garces, et al.
Publicado: (2026)

I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
por: Liu, Yuhang, et al.
Publicado: (2025)

Mechanics of Next Token Prediction with Self-Attention
por: Li, Yingcong, et al.
Publicado: (2024)

Uncovering Gaps in How Humans and LLMs Interpret Subjective Language
por: Jones, Erik, et al.
Publicado: (2025)

Dynamics of Spontaneous Topic Changes in Next Token Prediction with Self-Attention
por: Jia, Mumin, et al.
Publicado: (2025)

Auto-Regressive Next-Token Predictors are Universal Learners
por: Malach, Eran
Publicado: (2023)

Quantum Learning Theory Beyond Batch Binary Classification
por: Mohan, Preetham, et al.
Publicado: (2023)

Distribution-Free Robust Predict-Then-Optimize in Function Spaces
por: Patel, Yash, et al.
Publicado: (2026)

When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming
por: Corielli, Francesco
Publicado: (2026)

A Computationally Efficient Algorithm for Infinite-Horizon Average-Reward Linear MDPs
por: Hong, Kihyuk, et al.
Publicado: (2025)

How Different Tokenization Algorithms Impact LLMs and Transformer Models for Binary Code Analysis
por: Mostafa, Ahmed, et al.
Publicado: (2025)

Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy
por: Viswanathan, Karthik, et al.
Publicado: (2025)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
por: Shao, Chenze, et al.
Publicado: (2024)

Online Classification with Predictions
por: Raman, Vinod, et al.
Publicado: (2024)

SENTRA: Selected-Next-Token Transformer for LLM Text Detection
por: Plyler, Mitchell, et al.
Publicado: (2025)

Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
por: Rofin, Mark, et al.
Publicado: (2026)

NextLocLLM: Location Semantics Modeling and Coordinate-Based Next Location Prediction with LLMs
por: Liu, Shuai, et al.
Publicado: (2024)

LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
por: Zhou, Enshuai, et al.
Publicado: (2026)

Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement
por: Zhong, Qimin, et al.
Publicado: (2026)

Reject Only Critical Tokens: Pivot-Aware Speculative Decoding
por: Ziashahabi, Amir, et al.
Publicado: (2025)