Cover Image

Future Lens: Anticipating Subsequent Tokens from a Single Hidden State

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pal, Koyena, Sun, Jiuding, Yuan, Andrew, Wallace, Byron C., Bau, David
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
by: Feucht, Sheridan, et al.
Published: (2024)

Vector Arithmetic in Concept and Token Subspaces
by: Feucht, Sheridan, et al.
Published: (2025)

Do explanations generalize across large reasoning models?
by: Pal, Koyena, et al.
Published: (2026)

Function Vectors in Large Language Models
by: Todd, Eric, et al.
Published: (2023)

Can SAEs reveal and mitigate racial biases of LLMs in healthcare?
by: Ahsan, Hiba, et al.
Published: (2025)

The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis
by: Mueller, Aaron, et al.
Published: (2024)

Compared to What? Baselines and Metrics for Counterfactual Prompting
by: Yang, Zihao, et al.
Published: (2026)

Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
by: Roy, Debjyoti Saha, et al.
Published: (2024)

Transfer Learning from Foundational Optimization Embeddings to Unsupervised SAT Representations
by: Pal, Koyena, et al.
Published: (2026)

VickreyFeedback: Cost-efficient Data Construction for Reinforcement Learning from Human Feedback
by: Zhang, Guoxi, et al.
Published: (2024)

Erasing Conceptual Knowledge from Language Models
by: Gandikota, Rohit, et al.
Published: (2024)

Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains
by: Ramprasad, Sanjana, et al.
Published: (2024)

Do Activation Verbalization Methods Convey Privileged Information?
by: Li, Millicent, et al.
Published: (2025)

SIFiD: Reassess Summary Factual Inconsistency Detection with LLM
by: Yang, Jiuding, et al.
Published: (2024)

HyperSteer: Activation Steering at Scale with Hypernetworks
by: Sun, Jiuding, et al.
Published: (2025)

GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
by: Krishna, Kundan, et al.
Published: (2024)

Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learning
by: Deng, Wenlong, et al.
Published: (2025)

Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
by: Samragh, Mohammad, et al.
Published: (2025)

Attamba: Attending To Multi-Token States
by: Akhauri, Yash, et al.
Published: (2024)

Rethinking Token Reduction for State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
by: Sun, Jiuding, et al.
Published: (2025)

Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance
by: Chen, Xinzhu, et al.
Published: (2026)

Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking
by: Prakash, Nikhil, et al.
Published: (2024)

Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
by: Yang, Jiuding, et al.
Published: (2024)

Efficient Joint Prediction of Multiple Future Tokens
by: Ahn, Kwangjun, et al.
Published: (2025)

FoNE: Precise Single-Token Number Embeddings via Fourier Features
by: Zhou, Tianyi, et al.
Published: (2025)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
by: Sun, Yu, et al.
Published: (2024)

Discovering Forbidden Topics in Language Models
by: Rager, Can, et al.
Published: (2025)

MambaByte: Token-free Selective State Space Model
by: Wang, Junxiong, et al.
Published: (2024)

Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction
by: Singer, Matthew, et al.
Published: (2026)

Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment
by: Vaugrante, Laurène, et al.
Published: (2026)

Rethinking LLM Memorization through the Lens of Adversarial Compression
by: Schwarzschild, Avi, et al.
Published: (2024)

Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection
by: Srivastav, Devansh, et al.
Published: (2026)

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction
by: Walker, Nicholas
Published: (2024)

STS: Efficient Sparse Attention with Speculative Token Sparsity
by: Xu, Ceyu, et al.
Published: (2026)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

Enhancing Latent Computation in Transformers with Latent Tokens
by: Sun, Yuchang, et al.
Published: (2025)

Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
by: Wang, Qianli, et al.
Published: (2025)

Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity
by: Kuratov, Yuri, et al.
Published: (2025)

In-Context Algebra
by: Todd, Eric, et al.
Published: (2025)