Image de couverture de livre

Do language models plan ahead for future tokens?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Wilson, Morris, John X., Levine, Lionel
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Visualizing token importance for black-box language models
par: Rauba, Paulius, et autres
Publié: (2025)

Prompt reinforcing for long-term planning of large language models
par: Lin, Hsien-Chin, et autres
Publié: (2025)

Where is the signal in tokenization space?
par: Geh, Renato Lui, et autres
Publié: (2024)

EigenBench: A Comparative Behavioral Measure of Value Alignment
par: Chang, Jonathn, et autres
Publié: (2025)

Extracting Prompts by Inverting LLM Outputs
par: Zhang, Collin, et autres
Publié: (2024)

On multi-token prediction for efficient LLM inference
par: Mehra, Somesh, et autres
Publié: (2025)

Is Sanskrit the most token-efficient language? A quantitative study using GPT, Gemini, and SentencePiece
par: Kumar, Anshul
Publié: (2026)

Do different prompting methods yield a common task representation in language models?
par: Davidson, Guy, et autres
Publié: (2025)

Language models are better than humans at next-token prediction
par: Shlegeris, Buck, et autres
Publié: (2022)

Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
par: Singh, Aaditya K., et autres
Publié: (2024)

The pitfalls of next-token prediction
par: Bachmann, Gregor, et autres
Publié: (2024)

Looking beyond the next token
par: Thankaraj, Abitha, et autres
Publié: (2025)

Byte-token Enhanced Language Models for Temporal Point Processes Analysis
par: Kong, Quyu, et autres
Publié: (2025)

A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs
par: Merchant, Humzah, et autres
Publié: (2025)

Shaping capabilities with token-level data filtering
par: Rathi, Neil, et autres
Publié: (2026)

Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations
par: Zhao, Yize, et autres
Publié: (2024)

Scaling Transformer to 1M tokens and beyond with RMT
par: Bulatov, Aydar, et autres
Publié: (2023)

Anchor function: a type of benchmark functions for studying language models
par: Zhang, Zhongwang, et autres
Publié: (2024)

Aligning language models with human preferences
par: Korbak, Tomasz
Publié: (2024)

Evaluating language models as risk scores
par: Cruz, André F., et autres
Publié: (2024)

CogBench: a large language model walks into a psychology lab
par: Coda-Forno, Julian, et autres
Publié: (2024)

AtteSTNet -- An attention and subword tokenization based approach for code-switched text hate speech detection
par: Shingi, Geet, et autres
Publié: (2021)

Large language models reorganize representational geometry during in-context learning
par: Xiong, Hua-Dong, et autres
Publié: (2026)

Amortizing intractable inference in large language models
par: Hu, Edward J., et autres
Publié: (2023)

Interpretable Next-token Prediction via the Generalized Induction Head
par: Kim, Eunji, et autres
Publié: (2024)

Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
par: Nagarajan, Vaishnavh, et autres
Publié: (2025)

Perturbed examples reveal invariances shared by language models
par: Rawal, Ruchit, et autres
Publié: (2023)

A mean teacher algorithm for unlearning of language models
par: Klochkov, Yegor
Publié: (2025)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
par: Kwek, Eugene, et autres
Publié: (2025)

The language of time: a language model perspective on time-series foundation models
par: Xie, Yi, et autres
Publié: (2025)

Machine-generated text detection prevents language model collapse
par: Drayson, George, et autres
Publié: (2025)

All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling
par: Marconato, Emanuele, et autres
Publié: (2024)

Essential-Web v1.0: 24T tokens of organized web data
par: AI, Essential, et autres
Publié: (2025)

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
par: Raposo, David, et autres
Publié: (2024)

Simple linear attention language models balance the recall-throughput tradeoff
par: Arora, Simran, et autres
Publié: (2024)

Just read twice: closing the recall gap for recurrent language models
par: Arora, Simran, et autres
Publié: (2024)

Zero-shot generation of synthetic neurosurgical data with large language models
par: Barr, Austin A., et autres
Publié: (2025)

Repetitions are not all alike: distinct mechanisms sustain repetition in language models
par: Mahaut, Matéo, et autres
Publié: (2025)

Only relative ranks matter in weight-clustered large language models
par: Aizpurua, Borja, et autres
Publié: (2026)

How do language models learn facts? Dynamics, curricula and hallucinations
par: Zucchet, Nicolas, et autres
Publié: (2025)