Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.06584 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909991356596224 |
|---|---|
| author | Glowacki, Maciej |
| author_facet | Glowacki, Maciej |
| contents | Learning robust and generalisable abstractions from high-dimensional input data is a central challenge in machine learning and its applications to high-energy physics (HEP). Solutions of lower functional complexity are known to produce abstractions that generalise more effectively and are more robust to input perturbations. In complex hypothesis spaces, inductive biases make such solutions learnable by shaping the loss geometry during optimisation. In a HEP classification task, we show that a soft symmetry respecting inductive bias creates approximate degeneracies in the loss, which we identify as pseudo-Goldstone modes. We quantify functional complexity using metrics derived from first principles Hessian analysis and via compressibility. Our results demonstrate that solutions of lower complexity give rise to abstractions that are more generalisable, robust, and efficiently distillable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_06584 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Softly Induced Functional Simplicity: Implications for Neural Network Generalisation, Robustness, and Distillation Glowacki, Maciej Machine Learning High Energy Physics - Experiment Learning robust and generalisable abstractions from high-dimensional input data is a central challenge in machine learning and its applications to high-energy physics (HEP). Solutions of lower functional complexity are known to produce abstractions that generalise more effectively and are more robust to input perturbations. In complex hypothesis spaces, inductive biases make such solutions learnable by shaping the loss geometry during optimisation. In a HEP classification task, we show that a soft symmetry respecting inductive bias creates approximate degeneracies in the loss, which we identify as pseudo-Goldstone modes. We quantify functional complexity using metrics derived from first principles Hessian analysis and via compressibility. Our results demonstrate that solutions of lower complexity give rise to abstractions that are more generalisable, robust, and efficiently distillable. |
| title | Softly Induced Functional Simplicity: Implications for Neural Network Generalisation, Robustness, and Distillation |
| topic | Machine Learning High Energy Physics - Experiment |
| url | https://arxiv.org/abs/2601.06584 |