A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Louie Hong, Li, Yuhao, Liu, Shengchao
Format:	Preprint
Published:	2026
Subjects:	Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach
by: Chicchi, Lorenzo, et al.
Published: (2024)

Spectral Architecture Search for Neural Network Models
by: Peri, Gianluca, et al.
Published: (2025)

Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications
by: Böttcher, Lucas, et al.
Published: (2024)

Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks
by: Di Carlo, Luca, et al.
Published: (2025)

Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise
by: Chicchi, Lorenzo, et al.
Published: (2024)

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks
by: Marino, Raffaele, et al.
Published: (2023)

Stable Attractors for Neural networks classification via Ordinary Differential Equations (SA-nODE)
by: Marino, Raffaele, et al.
Published: (2023)

Frustrations on Decorated Square Lattice in Ising Model
by: Kassan-Ogly, F. A., et al.
Published: (2024)

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems
by: Angelini, Maria Chiara, et al.
Published: (2023)

Deep Neural Nets as Hamiltonians
by: Winer, Mike, et al.
Published: (2025)

A Generative Neural Annealer for Black-Box Combinatorial Optimization
by: Zhang, Yuan-Hang, et al.
Published: (2025)

Emergent Slow Thinking in LLMs as Inverse Tree Freezing
by: Hu, Sihan, et al.
Published: (2025)

Spin glass model of in-context learning
by: Li, Yuhao, et al.
Published: (2024)

Gaussian Universality in Neural Network Dynamics with Generalized Structured Input Distributions
by: Bae, Jaeyong, et al.
Published: (2024)

Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis
by: Lee, Donghee, et al.
Published: (2024)

Biased Generalization in Diffusion Models
by: Garnier-Brun, Jerome, et al.
Published: (2026)

Rare Event Analysis of Large Language Models
by: Dorman, Jake McAllister, et al.
Published: (2026)

Explaining the effects of non-convergent sampling in the training of Energy-Based Models
by: Agoritsas, Elisabeth, et al.
Published: (2023)

An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models
by: Mao, Jialin, et al.
Published: (2025)

Message Passing Variational Autoregressive Network for Solving Intractable Ising Models
by: Ma, Qunlong, et al.
Published: (2024)

Decorated Clusters and Geometrical Frustration in Cluster Spin Glass: A Random Graph Approach
by: Magalhaes, S. G., et al.
Published: (2025)

Intuition emerges in Maximum Caliber models at criticality
by: Arola-Fernández, Lluís
Published: (2025)

Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models
by: Del Bono, Luca Maria, et al.
Published: (2025)

Demonstrating Real Advantage of Machine-Learning-Enhanced Monte Carlo for Combinatorial Optimization
by: Del Bono, Luca Maria, et al.
Published: (2025)

The critical slowing down in diffusion models
by: Del Bono, Luca Maria, et al.
Published: (2026)

Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers
by: Albrychiewicz, Emil, et al.
Published: (2026)

Analytic theory of dropout regularization
by: Mori, Francesco, et al.
Published: (2025)

The Copycat Perceptron: Smashing Barriers Through Collective Learning
by: Catania, Giovanni, et al.
Published: (2023)

Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines
by: Fachechi, Alberto, et al.
Published: (2024)

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems
by: Biazzo, Indaco
Published: (2023)

Distinct mechanisms underlying in-context learning in transformers
by: Gibson, Cole, et al.
Published: (2026)

Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
by: Mori, Francesco, et al.
Published: (2024)

Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks
by: Tamai, Keiichi, et al.
Published: (2023)

Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
by: Giorlandino, Alessio, et al.
Published: (2025)

Inferring Higher-Order Couplings with Neural Networks
by: Decelle, Aurélien, et al.
Published: (2025)

Discrete generative diffusion models without stochastic differential equations: a tensor network approach
by: Causer, Luke, et al.
Published: (2024)

Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
by: Giorlandino, Alessio, et al.
Published: (2026)

The Physics of Data and Tasks: Theories of Locality and Compositionality in Deep Learning
by: Favero, Alessandro
Published: (2025)

Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
by: Ghio, Davide, et al.
Published: (2023)

Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
by: Bocchi, Dario, et al.
Published: (2026)