Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929250081177600 |
|---|---|
| author | Viallard, Paul Emonet, Rémi Habrard, Amaury Morvant, Emilie Zantedeschi, Valentina |
| author_facet | Viallard, Paul Emonet, Rémi Habrard, Amaury Morvant, Emilie Zantedeschi, Valentina |
| contents | In statistical learning theory, a generalization bound usually involves a complexity measure imposed by the considered theoretical framework. This limits the scope of such bounds, as other forms of capacity measures or regularizations are used in algorithms. In this paper, we leverage the framework of disintegrated PAC-Bayes bounds to derive a general generalization bound instantiable with arbitrary complexity measures. One trick to prove such a result involves considering a commonly used family of distributions: the Gibbs distributions. Our bound stands in probability jointly over the hypothesis and the learning sample, which allows the complexity to be adapted to the generalization gap as it can be customized to fit both the hypothesis class and the task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13285 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures Viallard, Paul Emonet, Rémi Habrard, Amaury Morvant, Emilie Zantedeschi, Valentina Machine Learning In statistical learning theory, a generalization bound usually involves a complexity measure imposed by the considered theoretical framework. This limits the scope of such bounds, as other forms of capacity measures or regularizations are used in algorithms. In this paper, we leverage the framework of disintegrated PAC-Bayes bounds to derive a general generalization bound instantiable with arbitrary complexity measures. One trick to prove such a result involves considering a commonly used family of distributions: the Gibbs distributions. Our bound stands in probability jointly over the hypothesis and the learning sample, which allows the complexity to be adapted to the generalization gap as it can be customized to fit both the hypothesis class and the task. |
| title | Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.13285 |