Cover Image

Overfitting has a limitation: a model-independent generalization gap bound based on Rényi entropy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Suzuki, Atsushi, Wang, Jing
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Information Theory Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Foundation of Calculating Normalized Maximum Likelihood for Continuous Probability Models
by: Suzuki, Atsushi, et al.
Published: (2024)

Maximum entropy based testing in network models: ERGMs and constrained optimization
by: Ghosh, Subhro, et al.
Published: (2026)

Quantum Fisher information matrices from Rényi relative entropies
by: Wilde, Mark M.
Published: (2025)

Graph Attention Network for Node Regression on Random Geometric Graphs with Erdős--Rényi contamination
by: Laha, Somak, et al.
Published: (2026)

Estimation of discrete distributions in relative entropy, and the deviations of the missing mass
by: Mourtada, Jaouad
Published: (2025)

A unified recipe for deriving (time-uniform) PAC-Bayes bounds
by: Chugg, Ben, et al.
Published: (2023)

Generalization of LiNGAM that allows confounding
by: Suzuki, Joe, et al.
Published: (2024)

Fundamental limits of distributed covariance matrix estimation via a conditional strong data processing inequality
by: Rahmani, Mohammad Reza, et al.
Published: (2025)

Theoretical limits of descending $\ell_0$ sparse-regression ML algorithms
by: Stojnic, Mihailo
Published: (2024)

Towards a mathematical theory for consistency training in diffusion models
by: Li, Gen, et al.
Published: (2024)

Random pairing MLE for estimation of item parameters in Rasch model
by: Yang, Yuepeng, et al.
Published: (2024)

Ridge interpolators in correlated factor regression models -- exact risk analysis
by: Stojnic, Mihailo
Published: (2024)

A new pathway to generative artificial intelligence by minimizing the maximum entropy
by: Miotto, Mattia, et al.
Published: (2025)

Fundamental limits of community detection from multi-view data: multi-layer, dynamic and partially labeled block models
by: Yang, Xiaodong, et al.
Published: (2024)

Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions
by: Zhang, Kaihong, et al.
Published: (2024)

Spectral Ranking Inferences based on General Multiway Comparisons
by: Fan, Jianqing, et al.
Published: (2023)

Top-$K$ ranking with a monotone adversary
by: Yang, Yuepeng, et al.
Published: (2024)

Instance-Optimal Estimation with Multiple LLM Judges on a Budget
by: Lee, Junghyun, et al.
Published: (2026)

Consistent model selection in the spiked Wigner model via AIC-type criteria
by: Mukherjee, Soumendu Sundar
Published: (2023)

The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model
by: Shi, Laixi, et al.
Published: (2023)

Precise analysis of ridge interpolators under heavy correlations -- a Random Duality Theory view
by: Stojnic, Mihailo
Published: (2024)

Learning sparse generalized linear models with binary outcomes via iterative hard thresholding
by: Matsumoto, Namiko, et al.
Published: (2025)

Beyond Benign Overfitting in Nadaraya-Watson Interpolators
by: Barzilai, Daniel, et al.
Published: (2025)

On the Nonasymptotic Scaling Guarantee of Hyperparameter Estimation in Inhomogeneous, Weakly-Dependent Complex Network Dynamical Systems
by: Yu, Yi, et al.
Published: (2026)

Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting
by: Christensen, Ronald
Published: (2024)

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning
by: Kim, Gilhan
Published: (2026)

Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
by: Kim, Yeichan, et al.
Published: (2025)

Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions
by: Lu, Weihao, et al.
Published: (2026)

Benign Overfitting in Time Series Linear Models with Over-Parameterization
by: Nakakita, Shogo, et al.
Published: (2022)

Online Clustering of Data Sequences with Bandit Information
by: Chandran, G Dhinesh, et al.
Published: (2025)

Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models
by: Chen, Haohua, et al.
Published: (2025)

On Robust Hypothesis Testing with respect to the Hellinger Distance
by: Modak, Eeshan, et al.
Published: (2025)

Distribution free M-estimation
by: Areces, Felipe, et al.
Published: (2025)

Statistical Mean Estimation with Coded Relayed Observations
by: Ling, Yan Hao, et al.
Published: (2025)

A Unified Representation of Density-Power-Based Divergences Reducible to M-Estimation
by: Kobayashi, Masahiro
Published: (2025)

Sequential 1-bit Mean Estimation with Near-Optimal Sample Complexity
by: Lau, Ivan, et al.
Published: (2025)

Asymptotic Theory of Eigenvectors for Latent Embeddings with Generalized Laplacian Matrices
by: Fan, Jianqing, et al.
Published: (2025)

Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal
by: Yuan, Deheng, et al.
Published: (2025)

Robust Estimation Under Heterogeneous Corruption Rates
by: Chaudhuri, Syomantak, et al.
Published: (2025)

On Instability of Minimax Optimal Optimism-Based Bandit Algorithms
by: Praharaj, Samya, et al.
Published: (2025)