Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2026
|
| Online Access: | https://doi.org/10.5281/zenodo.19645106 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866901604903419904 |
|---|---|
| author | Chawla, Aman |
| author_facet | Chawla, Aman |
| contents | <p>Robustness in differentiable models is often pursued through smooth aggregation mechanisms or adversarially<br>motivated training objectives. Such approaches can improve output stability, but they also tend to preserve a<br>single fully differentiable computation through which gradients remain coherent. This creates the specific<br>structural question studied here: whether forward stabilization can be introduced without forcing backward<br>propagation to follow the same aggregated pathway.<br>We study this question through a stochastic transformer-style prototype in which each realization uses<br>one-path attention, forward outputs are averaged across samples, and gradients are restricted to a single<br>sampled realization by a stop-gradient surrogate. The purpose of the construction is not to claim benchmarkscale<br>robustness, but to isolate the effect of decoupling forward aggregation from backward propagation in a<br>controlled setting. This yields a concrete mechanism for testing whether stability and gradient coherence can<br>be varied separately.<br>In the reference experiment implemented by transformerNRRv17.py, increasing the number of forward<br>samples reduces adversarial output sensitivity from 21.47 at = 1 to 6.91 at = 20, while the measured<br>gradient-cosine statistic remains near 0.81. Within this prototype, the results support the narrower claim that<br>output averaging can improve stability without producing a corresponding rise in gradient alignment. The<br>broader significance is therefore provisional but structural: the placement of aggregation relative to gradient<br>flow appears to be a meaningful design axis for future robustness work.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19645106 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Decoupling Forward Aggregation and Backward Propagation for Robust Neural Computation: a Python Study Chawla, Aman <p>Robustness in differentiable models is often pursued through smooth aggregation mechanisms or adversarially<br>motivated training objectives. Such approaches can improve output stability, but they also tend to preserve a<br>single fully differentiable computation through which gradients remain coherent. This creates the specific<br>structural question studied here: whether forward stabilization can be introduced without forcing backward<br>propagation to follow the same aggregated pathway.<br>We study this question through a stochastic transformer-style prototype in which each realization uses<br>one-path attention, forward outputs are averaged across samples, and gradients are restricted to a single<br>sampled realization by a stop-gradient surrogate. The purpose of the construction is not to claim benchmarkscale<br>robustness, but to isolate the effect of decoupling forward aggregation from backward propagation in a<br>controlled setting. This yields a concrete mechanism for testing whether stability and gradient coherence can<br>be varied separately.<br>In the reference experiment implemented by transformerNRRv17.py, increasing the number of forward<br>samples reduces adversarial output sensitivity from 21.47 at = 1 to 6.91 at = 20, while the measured<br>gradient-cosine statistic remains near 0.81. Within this prototype, the results support the narrower claim that<br>output averaging can improve stability without producing a corresponding rise in gradient alignment. The<br>broader significance is therefore provisional but structural: the placement of aggregation relative to gradient<br>flow appears to be a meaningful design axis for future robustness work.</p> |
| title | Decoupling Forward Aggregation and Backward Propagation for Robust Neural Computation: a Python Study |
| url | https://doi.org/10.5281/zenodo.19645106 |