Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2026
|
| Online Access: | https://doi.org/10.5281/zenodo.19396270 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866901123764322304 |
|---|---|
| author | tamanaganti |
| author_facet | tamanaganti |
| contents | <h2>Changes from v1.0</h2> <h3>Fixed</h3> <ul> <li>The §6 verification snippet in <code>REPRODUCE.md</code> now takes <strong><code>bri_imagenet_c</code> from <code>generalization_results.json</code></strong>, the same artifact that holds all 15 models' shift metrics. Previously it merged BRI from <code>imagenet_c_summary.json</code> while reading outcomes from the generalization file, which is easy to get out of sync and is unnecessary for a headline check. <strong>Published correlation values are unchanged</strong>—only the verification script was corrected.</li> </ul> <h3>Added</h3> <ul> <li><strong>Kendall τ</strong> alongside Spearman in the correlation snippet, with expected values aligned to <code>scipy.stats.kendalltau</code> on the bundled JSON (e.g. ImageNet-R τ = −0.390, p = 0.046; ImageNet-A τ = +0.448, p = 0.021; NINCO τ = +0.619, p ≈ 0.001). Signs and significance match Spearman.</li> <li><strong>Pre-flight</strong> Python snippet to confirm <code>bri_imagenet_c</code> is non-null for every model record before correlating.</li> <li><strong>Instructions</strong> for re-running <code>10_generalization_eval_all15.py</code> without stale <code>*_generalization.json</code> cache issues (run Script 04 first; clear caches when regenerating).</li> </ul> <h3>Unchanged</h3> <ul> <li>Application code under <code>src/</code> and <code>scripts/</code></li> <li>Committed result files and figures</li> <li>Reported statistics in the paper</li> </ul> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19396270 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | tamanaganti/abtp: Reproducibility fixes and Kendall τ verification tamanaganti <h2>Changes from v1.0</h2> <h3>Fixed</h3> <ul> <li>The §6 verification snippet in <code>REPRODUCE.md</code> now takes <strong><code>bri_imagenet_c</code> from <code>generalization_results.json</code></strong>, the same artifact that holds all 15 models' shift metrics. Previously it merged BRI from <code>imagenet_c_summary.json</code> while reading outcomes from the generalization file, which is easy to get out of sync and is unnecessary for a headline check. <strong>Published correlation values are unchanged</strong>—only the verification script was corrected.</li> </ul> <h3>Added</h3> <ul> <li><strong>Kendall τ</strong> alongside Spearman in the correlation snippet, with expected values aligned to <code>scipy.stats.kendalltau</code> on the bundled JSON (e.g. ImageNet-R τ = −0.390, p = 0.046; ImageNet-A τ = +0.448, p = 0.021; NINCO τ = +0.619, p ≈ 0.001). Signs and significance match Spearman.</li> <li><strong>Pre-flight</strong> Python snippet to confirm <code>bri_imagenet_c</code> is non-null for every model record before correlating.</li> <li><strong>Instructions</strong> for re-running <code>10_generalization_eval_all15.py</code> without stale <code>*_generalization.json</code> cache issues (run Script 04 first; clear caches when regenerating).</li> </ul> <h3>Unchanged</h3> <ul> <li>Application code under <code>src/</code> and <code>scripts/</code></li> <li>Committed result files and figures</li> <li>Reported statistics in the paper</li> </ul> |
| title | tamanaganti/abtp: Reproducibility fixes and Kendall τ verification |
| url | https://doi.org/10.5281/zenodo.19396270 |