Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910200307384320 |
|---|---|
| author | Kratsios, Anastasis Cousins, Gregory Borde, Haitz Sáez de Ocáriz Kim, Bum Jun Brugiapaglia, Simone |
| author_facet | Kratsios, Anastasis Cousins, Gregory Borde, Haitz Sáez de Ocáriz Kim, Bum Jun Brugiapaglia, Simone |
| contents | We show that, in a precise sense, a broad class of feedforward neural networks learn (have finite sample complexity) in the PAC model: every fixed finite feedforward architecture whose layers are definable in an o-minimal structure has finite sample complexity in the agnostic PAC setting, even with unbounded parameters. This covers standard fixed-size MLPs, CNNs, GNNs, and transformers with fixed sequence length, together with the operations and layers typically used in such architectures, including linear projections, residual connections, attention mechanisms, pooling layers, normalization layers, and admissible positional encodings. Hence, distribution-free learnability for modern non-recurrent architectures is not an exceptional property of particular activations or architecture-specific VC arguments, but a consequence of tame feedforward computation. Our results reposition finite-sample PAC learnability as a baseline rather than a differentiator: they shift the focus of architectural comparison toward inductive biases, symmetries and geometric priors, scalability, and optimization behaviour. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_07097 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity Kratsios, Anastasis Cousins, Gregory Borde, Haitz Sáez de Ocáriz Kim, Bum Jun Brugiapaglia, Simone Machine Learning Neural and Evolutionary Computing Logic Statistics Theory 68Q32, 03C64, 03C98, 68T07, 62G05 We show that, in a precise sense, a broad class of feedforward neural networks learn (have finite sample complexity) in the PAC model: every fixed finite feedforward architecture whose layers are definable in an o-minimal structure has finite sample complexity in the agnostic PAC setting, even with unbounded parameters. This covers standard fixed-size MLPs, CNNs, GNNs, and transformers with fixed sequence length, together with the operations and layers typically used in such architectures, including linear projections, residual connections, attention mechanisms, pooling layers, normalization layers, and admissible positional encodings. Hence, distribution-free learnability for modern non-recurrent architectures is not an exceptional property of particular activations or architecture-specific VC arguments, but a consequence of tame feedforward computation. Our results reposition finite-sample PAC learnability as a baseline rather than a differentiator: they shift the focus of architectural comparison toward inductive biases, symmetries and geometric priors, scalability, and optimization behaviour. |
| title | Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity |
| topic | Machine Learning Neural and Evolutionary Computing Logic Statistics Theory 68Q32, 03C64, 03C98, 68T07, 62G05 |
| url | https://arxiv.org/abs/2605.07097 |