Cover Image

Multiple Token Divergence: Measuring and Steering In-Context Computation Density

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Herrmann, Vincent, Alcaide, Eric, Wand, Michael, Schmidhuber, Jürgen
Format:	Preprint
Published:	2025
Subjects:	Machine Learning I.2.6
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Measuring In-Context Computation Complexity via Hidden State Prediction
by: Herrmann, Vincent, et al.
Published: (2025)

Learning Useful Representations of Recurrent Neural Network Weight Matrices
by: Herrmann, Vincent, et al.
Published: (2024)

Interestingness as an Inductive Heuristic for Future Compression Progress
by: Herrmann, Vincent, et al.
Published: (2026)

Upside Down Reinforcement Learning with Policy Generators
by: Di Ventura, Jacopo, et al.
Published: (2025)

Fusing Rewards and Preferences in Reinforcement Learning
by: Khorasani, Sadegh, et al.
Published: (2025)

Towards a Robust Soft Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms
by: Alhakami, Mohannad, et al.
Published: (2024)

Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning
by: Wang, Yuhui, et al.
Published: (2024)

E$^2$M: Double Bounded $α$-Divergence Optimization for Tensor-based Discrete Density Estimation
by: Ghalamkari, Kazu, et al.
Published: (2024)

Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data
by: Sergazinov, Renat, et al.
Published: (2025)

Token Space: A Category Theory Framework for AI Computations
by: Pan, Wuming
Published: (2024)

2Mamba2Furious: Linear in Complexity, Competitive in Accuracy
by: Mongaras, Gabriel, et al.
Published: (2026)

Before the Last Token: Diagnosing Final-Token Safety Probe Failures
by: Doda, Shravan
Published: (2026)

Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
by: Chapman, James, et al.
Published: (2025)

Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models
by: Huang, Yunpeng, et al.
Published: (2024)

Measuring IIA Violations in Similarity Choices with Bayesian Models
by: Corrêa, Hugo Sales, et al.
Published: (2025)

ZClassifier: Temperature Tuning and Manifold Approximation via KL Divergence on Logit Space
by: Yong, Shim Soon
Published: (2025)

Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features
by: Cho, Seonglae, et al.
Published: (2026)

Deep Memory Search: A Metaheuristic Approach for Optimizing Heuristic Search
by: Hedar, Abdel-Rahman, et al.
Published: (2024)

Latent Instruction Representation Alignment: defending against jailbreaks, backdoors and undesired knowledge in LLMs
by: Easley, Eric, et al.
Published: (2026)

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction
by: Walker, Nicholas
Published: (2024)

Scaling Laws for Neural Material Models
by: Trikha, Akshay, et al.
Published: (2025)

Automatic Album Sequencing
by: Herrmann, Vincent, et al.
Published: (2024)

Pruning Spurious Subgraphs for Graph Out-of-Distribution Generalization
by: Yao, Tianjun, et al.
Published: (2025)

Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization
by: Dai, Yanning, et al.
Published: (2026)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)

FlowSteer: Towards Agents Designing Agentic Workflows via Reinforced Progressive Canvas Editing
by: Zhang, Mingda, et al.
Published: (2026)

AI and Machine Learning Approaches for Predicting Nanoparticles Toxicity The Critical Role of Physiochemical Properties
by: Yousaf, Iqra
Published: (2024)

CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
by: Cho, Seonglae, et al.
Published: (2025)

Dynamic Attentional Context Scoping: Agent-Triggered Focus Sessions for Isolated Per-Agent Steering in Multi-Agent LLM Orchestration
by: Patel, Nickson
Published: (2026)

Rethinking Thinking Tokens: Understanding Why They Underperform in Practice
by: Vennam, Sreeram, et al.
Published: (2024)

Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension
by: Herde, Marek, et al.
Published: (2024)

Implicit Counterfactual Data Augmentation for Robust Learning
by: Zhou, Xiaoling, et al.
Published: (2023)

On Divergence Measures for Training GFlowNets
by: da Silva, Tiago, et al.
Published: (2024)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
by: He, Yanjin, et al.
Published: (2025)

Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
by: Shravan, Rohan
Published: (2026)

Kernel Density Matrices for Probabilistic Deep Learning
by: González, Fabio A., et al.
Published: (2023)

I-GLIDE: Input Groups for Latent Health Indicators in Degradation Estimation
by: Thil, Lucas, et al.
Published: (2025)

FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models
by: Polly, Fabien
Published: (2026)

Quantifying the Energy Floor: Direct Measurement and Replay Buffer Bias in SAC-Based HVAC Control on sbsim
by: Li, Bo, et al.
Published: (2026)

A Simple Generalisation of the Implicit Dynamics of In-Context Learning
by: Innocenti, Francesco, et al.
Published: (2025)