OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918496389038080 |
|---|---|
| author | Fernández-Hernández, Alberto Mestre, Jose I. Pérez-Corral, Cristian Dolz, Manuel F. Duato, Jose Quintana-Ortí, Enrique S. |
| author_facet | Fernández-Hernández, Alberto Mestre, Jose I. Pérez-Corral, Cristian Dolz, Manuel F. Duato, Jose Quintana-Ortí, Enrique S. |
| contents | Activation functions are what make deep networks expressive: without them, the model collapses to a linear map. Yet we still evaluate training mostly from the outside, through loss, accuracy, return, or final calibration, while the internal structural evolution of the network remains largely unobserved. In this paper, we argue that the Overfitting--Underfitting Indicator (OUI) should be understood as a first practical observable of that internal structure. Across our recent results, OUI consistently appears as an early, label-free, activation-based signal that reveals whether a network is entering a poor or promising training regime before convergence. In supervised learning, it anticipates weight decay regimes; in reinforcement learning, it discriminates learning-rate regimes early in PPO actor--critic; and in online control, it can drive layer-wise weight decay adaptation. Read together with recent evidence that activation patterns tend to stabilize earlier than parameters, these results suggest a broader research direction: an activation-centric theory of training dynamics. OUI is becoming an empirical foothold toward this theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_11570 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training Fernández-Hernández, Alberto Mestre, Jose I. Pérez-Corral, Cristian Dolz, Manuel F. Duato, Jose Quintana-Ortí, Enrique S. Machine Learning Activation functions are what make deep networks expressive: without them, the model collapses to a linear map. Yet we still evaluate training mostly from the outside, through loss, accuracy, return, or final calibration, while the internal structural evolution of the network remains largely unobserved. In this paper, we argue that the Overfitting--Underfitting Indicator (OUI) should be understood as a first practical observable of that internal structure. Across our recent results, OUI consistently appears as an early, label-free, activation-based signal that reveals whether a network is entering a poor or promising training regime before convergence. In supervised learning, it anticipates weight decay regimes; in reinforcement learning, it discriminates learning-rate regimes early in PPO actor--critic; and in online control, it can drive layer-wise weight decay adaptation. Read together with recent evidence that activation patterns tend to stabilize earlier than parameters, these results suggest a broader research direction: an activation-centric theory of training dynamics. OUI is becoming an empirical foothold toward this theory. |
| title | OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.11570 |