Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917537162199040 |
|---|---|
| author | Liu, Yanxiao Fan, Yijun Gündüz, Deniz |
| author_facet | Liu, Yanxiao Fan, Yijun Gündüz, Deniz |
| contents | Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07999 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities Liu, Yanxiao Fan, Yijun Gündüz, Deniz Information Theory Machine Learning Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses. |
| title | Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities |
| topic | Information Theory Machine Learning |
| url | https://arxiv.org/abs/2602.07999 |