Image de couverture de livre

Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Viswanathan, Karthik, Park, Sang Eon
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Statistical Mechanics Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Berezinskii--Kosterlitz--Thouless transition in a context-sensitive random language model
par: Toji, Yuma, et autres
Publié: (2024)

Phase transition on a context-sensitive random language model with short range interactions
par: Toji, Yuma, et autres
Publié: (2026)

The Information Geometry of Softmax: Probing and Steering
par: Park, Kiho, et autres
Publié: (2026)

Phase Transitions in the Output Distribution of Large Language Models
par: Arnold, Julian, et autres
Publié: (2024)

The Geometry of Tokens in Internal Representations of Large Language Models
par: Viswanathan, Karthik, et autres
Publié: (2025)

$α$-divergence Improves the Entropy Production Estimation via Machine Learning
par: Kwon, Euijoon, et autres
Publié: (2023)

Effects of structural properties of neural networks on machine learning performance
par: Arya, Yash, et autres
Publié: (2025)

Neural Entropy
par: Premkumar, Akhil
Publié: (2024)

Mapping of attention mechanisms to a generalized Potts model
par: Rende, Riccardo, et autres
Publié: (2023)

How transformers learn structured data: insights from hierarchical filtering
par: Garnier-Brun, Jerome, et autres
Publié: (2024)

Metropolis Monte Carlo sampling: convergence, localization transition and optimality
par: Chepelianskii, Alexei D., et autres
Publié: (2022)

Hessian Geometry of Latent Space in Generative Models
par: Lobashev, Alexander, et autres
Publié: (2025)

Universal Approximation of Mean-Field Models via Transformers
par: Biswal, Shiba, et autres
Publié: (2024)

Coupled Entropy: A Goldilocks Generalization for Complex Systems
par: Nelson, Kenric P.
Publié: (2025)

An Optimized Franz-Parisi Criterion and its Equivalence with SQ Lower Bounds
par: Chen, Siyu, et autres
Publié: (2025)

The monotonicity of the Franz-Parisi potential is equivalent with Low-degree MMSE lower bounds
par: Tsirkas, Konstantinos, et autres
Publié: (2026)

Riemannian Stochastic Interpolants for Amorphous Particle Systems
par: Grenioux, Louis, et autres
Publié: (2025)

Fast, Modular, and Differentiable Framework for Machine Learning-Enhanced Molecular Simulations
par: Christiansen, Henrik, et autres
Publié: (2025)

Controlling dynamics of stochastic systems with deep reinforcement learning
par: Mukhamadiarov, Ruslan
Publié: (2025)

Estimating Solvation Free Energies with Boltzmann Generators
par: Schebek, Maximilian, et autres
Publié: (2025)

Model selection for stochastic dynamics: a parsimonious and principled approach
par: Gerardos, Andonis
Publié: (2025)

Boltzmann Generators for Condensed Matter via Riemannian Flow Matching
par: Hoffmann, Emil, et autres
Publié: (2026)

Autonomous Discovery of the Ising Model's Critical Parameters with Reinforcement Learning
par: Man, Hai, et autres
Publié: (2026)

Accurate generation of stochastic dynamics based on multi-model Generative Adversarial Networks
par: Lanzoni, Daniele, et autres
Publié: (2023)

AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning
par: Kanaujia, Vikas, et autres
Publié: (2024)

Conditioning on a Volatility Proxy Compresses the Apparent Timescale of Collective Market Correlation
par: Bi, Yuda, et autres
Publié: (2026)

Differentiable free energy surface: a variational approach to directly observing rare events using generative deep-learning models
par: Li, Shuo-Hui, et autres
Publié: (2026)

A Theory of LLM Information Susceptibility
par: Song, Zhuo-Yang, et autres
Publié: (2026)

Softmax Attention with Constant Cost per Token
par: Heinsen, Franz A.
Publié: (2024)

Multi-Mode Quantum Annealing for Generative Representation Learning with Boltzmann Priors
par: Kim, Gilhan, et autres
Publié: (2026)

Quantum Neural Estimation of Entropies
par: Goldfeld, Ziv, et autres
Publié: (2023)

Speech perception: a model of word recognition
par: Luck, Jean-Marc, et autres
Publié: (2024)

Learning conformational ensembles of proteins based on backbone geometry
par: Wolf, Nicolas, et autres
Publié: (2025)

Nonlinear thermodynamic computing out of equilibrium
par: Whitelam, Stephen, et autres
Publié: (2024)

Implicit Binarization via Complex Phase Dynamics in Combinatorial Optimization
par: Cohen, Khen, et autres
Publié: (2026)

Statistics of Min-max Normalized Eigenvalues in Random Matrices
par: Nakada, Hyakka, et autres
Publié: (2025)

Laws of thermodynamics for exponential families
par: Balsubramani, Akshay
Publié: (2025)

Combined Representation and Generation with Diffusive State Predictive Information Bottleneck
par: John, Richard, et autres
Publié: (2025)

Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning
par: Elliott, Chris, et autres
Publié: (2026)

Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory
par: Watanabe, Sumio
Publié: (2024)