| _version_ | 1866901288409628672 |
|---|---|
| author | Anonymous |
| author_facet | Anonymous |
| contents | <div>Trained transformer checkpoints accompanying the paper "The Canary in the Carry Chain: Transformers Know the Schedule Before They Can Execute". The paper studies when a transformer that fails on an iterative algorithmic factored as $y = E(x, c(x))$ for a discrete controller $c(x)$ and an executor $E.$ </div> <div> </div> <div>This bundle contains the locally trained PyTorch state dicts that produced the published numbers in Sections 5, 6, 8, and 9 of the paper. Loadable with <code>torch.load(path, map_location='cpu')</code>.</div> <div> </div> <div>Included:</div> <div> <div>- <code>output_mps/b32/</code>: the main $3x+1$ base-32 <code>seq2seq</code> encoder (4 encoder + 1 decoder layers, d=512, 8 heads, FFN 2048, max length 16, 1000 epochs). This is the canonical interpretability subject. Every probe selectivity number, every MLP ablation row, and every cross-attention-mass and gradient-attribution number is computed on this checkpoint. Includes <code>ck_0010.pt</code> through <code>ck_0050.pt</code> and <code>best.pt</code>.</div> <div>- <code>output_do/</code>: the GPT-style 5-layer causal decoder-only replication used in Section 10 of the paper. Same width, format <code>[BOS]</code> $n$ <code>[SEP]</code> $\kappa(n)$ <code>[EOS]</code>, 300 epochs.</div> <div>- <code>output_ctrl/</code>: the single-permutation-orbit control model used in the task matrix.</div> <div>- <code>results_final/output_eci_seeds/baseline_s123/</code>: one seed of the 500-epoch baseline ECI replication.</div> </div> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_20017927 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | The Canary in the Carry Chain: model checkpoints for the long Collatz step Anonymous <div>Trained transformer checkpoints accompanying the paper "The Canary in the Carry Chain: Transformers Know the Schedule Before They Can Execute". The paper studies when a transformer that fails on an iterative algorithmic factored as $y = E(x, c(x))$ for a discrete controller $c(x)$ and an executor $E.$ </div> <div> </div> <div>This bundle contains the locally trained PyTorch state dicts that produced the published numbers in Sections 5, 6, 8, and 9 of the paper. Loadable with <code>torch.load(path, map_location='cpu')</code>.</div> <div> </div> <div>Included:</div> <div> <div>- <code>output_mps/b32/</code>: the main $3x+1$ base-32 <code>seq2seq</code> encoder (4 encoder + 1 decoder layers, d=512, 8 heads, FFN 2048, max length 16, 1000 epochs). This is the canonical interpretability subject. Every probe selectivity number, every MLP ablation row, and every cross-attention-mass and gradient-attribution number is computed on this checkpoint. Includes <code>ck_0010.pt</code> through <code>ck_0050.pt</code> and <code>best.pt</code>.</div> <div>- <code>output_do/</code>: the GPT-style 5-layer causal decoder-only replication used in Section 10 of the paper. Same width, format <code>[BOS]</code> $n$ <code>[SEP]</code> $\kappa(n)$ <code>[EOS]</code>, 300 epochs.</div> <div>- <code>output_ctrl/</code>: the single-permutation-orbit control model used in the task matrix.</div> <div>- <code>results_final/output_eci_seeds/baseline_s123/</code>: one seed of the 500-epoch baseline ECI replication.</div> </div> |
| title | The Canary in the Carry Chain: model checkpoints for the long Collatz step |
| url | https://doi.org/10.5281/zenodo.20017927 |