Cover Image

The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Takanami, Kaito, Takahashi, Takashi, Sakata, Ayaka
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Disordered Systems and Neural Networks
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Linear Regression with Low-Rank Tasks in-Context
by: Takanami, Kaito, et al.
Published: (2025)

Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing
by: Gu, Xiaosi, et al.
Published: (2025)

The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data
by: Takahashi, Takashi
Published: (2022)

Variational Gaussian Approximation in Replica Analysis of Parametric Models
by: Takahashi, Takashi
Published: (2025)

High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator
by: Ichikawa, Yuma, et al.
Published: (2025)

A replica analysis of under-bagging
by: Takahashi, Takashi
Published: (2024)

Asymptotic Dynamics of Alternating Minimization for Bilinear Regression
by: Okajima, Koki, et al.
Published: (2024)

Optimal Spectral Transitions in High-Dimensional Multi-Index Models
by: Defilippis, Leonardo, et al.
Published: (2025)

Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model
by: Bordelon, Blake, et al.
Published: (2026)

Dataset-Free Weight-Initialization on Restricted Boltzmann Machine
by: Yasuda, Muneki, et al.
Published: (2024)

Semi-analytic approximate stability selection for correlated data in generalized linear models
by: Takahashi, Takashi, et al.
Published: (2020)

Physics-inspired transformer quantum states via latent imaginary-time evolution
by: Yamazaki, Kimihiro, et al.
Published: (2026)

Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions
by: Keup, Christian, et al.
Published: (2024)

No Free Lunch From Random Feature Ensembles: Scaling Laws and Near-Optimality Conditions
by: Ruben, Benjamin S., et al.
Published: (2024)

Dynamical Mean-Field Theory of Self-Attention Neural Networks
by: Poc-López, Ángel, et al.
Published: (2024)

Self-attention as an attractor network: transient memories without backpropagation
by: D'Amico, Francesco, et al.
Published: (2024)

Learning Curves for Noisy Heterogeneous Feature-Subsampled Ridge Ensembles
by: Ruben, Benjamin S., et al.
Published: (2023)

Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos
by: Sarmiento, Lucas Fernandez
Published: (2026)

Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning
by: Staats, Max, et al.
Published: (2023)

The Quantization Model of Neural Scaling
by: Michaud, Eric J., et al.
Published: (2023)

Dynamical Regimes of Multimodal Diffusion Models
by: Albrychiewicz, Emil, et al.
Published: (2026)

A Generative Diffusion Model for Amorphous Materials
by: Yang, Kai, et al.
Published: (2025)

Critical Phase Transition in Large Language Models
by: Nakaishi, Kai, et al.
Published: (2024)

A Dynamical Model of Neural Scaling Laws
by: Bordelon, Blake, et al.
Published: (2024)

Emergence of Distortions in High-Dimensional Guided Diffusion Models
by: Ventura, Enrico, et al.
Published: (2026)

Soft Quantization: Model Compression Via Weight Coupling
by: Bernstein, Daniel T., et al.
Published: (2026)

Probing the Latent Hierarchical Structure of Data via Diffusion Models
by: Sclocchi, Antonio, et al.
Published: (2024)

The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
by: Nicoletti, Flavio, et al.
Published: (2026)

The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
by: Farné, Gabriele, et al.
Published: (2026)

Theory of Speciation Transitions in Diffusion Models with General Class Structure
by: Achilli, Beatrice, et al.
Published: (2026)

Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians
by: Tomihari, Akiyoshi, et al.
Published: (2025)

Class Imbalance in Anomaly Detection: Learning from an Exactly Solvable Model
by: Pezzicoli, F. S., et al.
Published: (2025)

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
by: Atanasov, Alexander, et al.
Published: (2025)

Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit
by: Meng, Lineghuan, et al.
Published: (2024)

EB-RANSAC: Random Sample Consensus based on Energy-Based Model
by: Yasuda, Muneki, et al.
Published: (2026)

Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model
by: Tabanelli, Hugo, et al.
Published: (2025)

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
by: Arnaboldi, Luca, et al.
Published: (2025)

Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
by: Bonnaire, Tony, et al.
Published: (2025)

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
by: Tanner, Kasimir, et al.
Published: (2024)

Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
by: Thériault, Robin, et al.
Published: (2024)