Bound by semanticity: universal laws governing the generalization-identification tradeoff
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909651738558464 |
|---|---|
| author | Nurisso, Marco Fernando, Jesseba Deshpande, Raj Perotti, Alan Marjieh, Raja Frankland, Steven M. Lewis, Richard L. Webb, Taylor W. Campbell, Declan Vaccarino, Francesco Cohen, Jonathan D. Petri, Giovanni |
| author_facet | Nurisso, Marco Fernando, Jesseba Deshpande, Raj Perotti, Alan Marjieh, Raja Frankland, Steven M. Lewis, Richard L. Webb, Taylor W. Campbell, Declan Vaccarino, Francesco Cohen, Jonathan D. Petri, Giovanni |
| contents | Intelligent systems must deploy internal representations that are simultaneously structured -- to support broad generalization -- and selective -- to preserve input identity. We expose a fundamental limit on this tradeoff. For any model whose representational similarity between inputs decays with finite semantic resolution $\varepsilon$, we derive closed-form expressions that pin its probability of correct generalization $p_S$ and identification $p_I$ to a universal Pareto front independent of input space geometry. Extending the analysis to noisy, heterogeneous spaces and to $n>2$ inputs predicts a sharp $1/n$ collapse of multi-input processing capacity and a non-monotonic optimum for $p_S$. A minimal ReLU network trained end-to-end reproduces these laws: during learning a resolution boundary self-organizes and empirical $(p_S,p_I)$ trajectories closely follow theoretical curves for linearly decaying similarity. Finally, we demonstrate that the same limits persist in two markedly more complex settings -- a convolutional neural network and state-of-the-art vision-language models -- confirming that finite-resolution similarity is a fundamental emergent informational constraint, not merely a toy-model artifact. Together, these results provide an exact theory of the generalization-identification trade-off and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14797 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bound by semanticity: universal laws governing the generalization-identification tradeoff Nurisso, Marco Fernando, Jesseba Deshpande, Raj Perotti, Alan Marjieh, Raja Frankland, Steven M. Lewis, Richard L. Webb, Taylor W. Campbell, Declan Vaccarino, Francesco Cohen, Jonathan D. Petri, Giovanni Machine Learning Artificial Intelligence Intelligent systems must deploy internal representations that are simultaneously structured -- to support broad generalization -- and selective -- to preserve input identity. We expose a fundamental limit on this tradeoff. For any model whose representational similarity between inputs decays with finite semantic resolution $\varepsilon$, we derive closed-form expressions that pin its probability of correct generalization $p_S$ and identification $p_I$ to a universal Pareto front independent of input space geometry. Extending the analysis to noisy, heterogeneous spaces and to $n>2$ inputs predicts a sharp $1/n$ collapse of multi-input processing capacity and a non-monotonic optimum for $p_S$. A minimal ReLU network trained end-to-end reproduces these laws: during learning a resolution boundary self-organizes and empirical $(p_S,p_I)$ trajectories closely follow theoretical curves for linearly decaying similarity. Finally, we demonstrate that the same limits persist in two markedly more complex settings -- a convolutional neural network and state-of-the-art vision-language models -- confirming that finite-resolution similarity is a fundamental emergent informational constraint, not merely a toy-model artifact. Together, these results provide an exact theory of the generalization-identification trade-off and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike. |
| title | Bound by semanticity: universal laws governing the generalization-identification tradeoff |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.14797 |