Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2026
|
| Subjects: | |
| Online Access: | https://doi.org/10.5281/zenodo.19223777 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- <p>The Order-Coupled Gradient Theorem establishes that each nonlinearity order has its own eective learning rate, governing the instantaneous rate of change per gradient step. A natural next question is whether one can write a closed-form ordinary dierential equation(ODE) for the per-order learning error in terms of the error variables alone, thereby obtaining complete training trajectories without tracking individual parameters. We answer this question negatively: we prove an irreducibility theorem showing that the time derivative of the per-order error cannot be expressed as any function of the collection of per-order errors. The same error value generically corresponds to dierent rates of change, because the per-neuron parameters carry directional information that scalar error quantities cannot recover. We then identify the teacherstudent setting as a special case where a closed ODE does existthe SaadSolla overlap ODEand verify it numerically. For the general case with arbitrary targets, we show that the exact instantaneous rate formula from the OCGT can be applied step-by-step from full parameter knowledge, achieving near-perfect one-step-ahead prediction and accurate Euler/trapezoidal trajectory integration. Along the way, we introduce the order spectrum decompositiona Parseval-type identity expressing any square-integrable function as a sum of per-order spectral energiesas the natural framework unifying these results.</p>