Cover Image

High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ichikawa, Yuma, Kashiwamura, Shuhei, Sakata, Ayaka
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Disordered Systems and Neural Networks Artificial Intelligence Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Statistical Mechanics of Min-Max Problems
by: Ichikawa, Yuma, et al.
Published: (2024)

Ratio Divergence Learning Using Target Energy in Restricted Boltzmann Machines: Beyond Kullback--Leibler Divergence Learning
by: Ishida, Yuichi, et al.
Published: (2024)

Limits of message passing for node classification: How class-bottlenecks restrict signal-to-noise ratio
by: Rubin, Jonathan, et al.
Published: (2025)

On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
by: Vilucchio, Matteo, et al.
Published: (2024)

The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model
by: Takanami, Kaito, et al.
Published: (2025)

Generalization Dynamics of Linear Diffusion Models
by: Merger, Claudia, et al.
Published: (2025)

A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
by: Yao, Louie Hong, et al.
Published: (2026)

Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing
by: Gu, Xiaosi, et al.
Published: (2025)

Kernel Density Estimators in Large Dimensions
by: Biroli, Giulio, et al.
Published: (2024)

Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach
by: Chicchi, Lorenzo, et al.
Published: (2024)

Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach
by: Adomaityte, Urte, et al.
Published: (2023)

High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality
by: Adomaityte, Urte, et al.
Published: (2023)

Spectral Architecture Search for Neural Network Models
by: Peri, Gianluca, et al.
Published: (2025)

Asymptotics of Learning with Deep Structured (Random) Features
by: Schröder, Dominik, et al.
Published: (2024)

Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications
by: Böttcher, Lucas, et al.
Published: (2024)

Variational Gaussian Approximation in Replica Analysis of Parametric Models
by: Takahashi, Takashi
Published: (2025)

Spectral Thresholds in Correlated Spiked Models and Fundamental Limits of Partial Least Squares
by: Mergny, Pierre, et al.
Published: (2025)

The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data
by: Takahashi, Takashi
Published: (2022)

Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks
by: Di Carlo, Luca, et al.
Published: (2025)

Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise
by: Chicchi, Lorenzo, et al.
Published: (2024)

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks
by: Marino, Raffaele, et al.
Published: (2023)

Stable Attractors for Neural networks classification via Ordinary Differential Equations (SA-nODE)
by: Marino, Raffaele, et al.
Published: (2023)

The Role of the Time-Dependent Hessian in High-Dimensional Optimization
by: Bonnaire, Tony, et al.
Published: (2024)

Tensor-Network Population Annealing
by: Oshima, Takumi, et al.
Published: (2026)

Asymmetric Scaling Laws from Sparse Features
by: Sous, John, et al.
Published: (2026)

High-dimensional manifold of solutions in neural networks: insights from statistical physics
by: Malatesta, Enrico M.
Published: (2023)

The Copycat Perceptron: Smashing Barriers Through Collective Learning
by: Catania, Giovanni, et al.
Published: (2023)

Thermal Min-Max Games: Unifying Bounded Rationality and Typical-Case Equilibrium
by: Ichikawa, Yuma
Published: (2026)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
by: Lauditi, Clarissa, et al.
Published: (2026)

A Generative Neural Annealer for Black-Box Combinatorial Optimization
by: Zhang, Yuan-Hang, et al.
Published: (2025)

A theoretical perspective on mode collapse in variational inference
by: Soletskyi, Roman, et al.
Published: (2024)

Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit
by: Meng, Lineghuan, et al.
Published: (2024)

Deep Neural Nets as Hamiltonians
by: Winer, Mike, et al.
Published: (2025)

Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics
by: Avidan, Yehonatan, et al.
Published: (2023)

Demonstrating Real Advantage of Machine-Learning-Enhanced Monte Carlo for Combinatorial Optimization
by: Del Bono, Luca Maria, et al.
Published: (2025)

Precise asymptotic analysis of Sobolev training for random feature models
by: Fisher, Katharine E, et al.
Published: (2025)

Generalization performance of narrow one-hidden layer networks in the teacher-student setting
by: Ortiz, Rodrigo Pérez, et al.
Published: (2025)

Gaussian Universality of Perceptrons with Random Labels
by: Gerace, Federica, et al.
Published: (2022)

Spectral Phase Transition and Optimal PCA in Block-Structured Spiked models
by: Mergny, Pierre, et al.
Published: (2024)

The twin peaks of learning neural networks
by: Demyanenko, Elizaveta, et al.
Published: (2024)