| _version_ | 1866902328651546624 |
|---|---|
| author | Abhinav Jain |
| author_facet | Abhinav Jain |
| contents | <h1>v1.0 — 20k Multi-Seed State vs Pixel PPO Benchmark</h1> <h2>Summary</h2> <p>This release marks the first reproducible benchmark of state-based vs pixel-based PPO on Hopper-v4 under controlled, identical hyperparameters.</p> <p>The goal of this repository is not to propose a new algorithm, but to quantify the representation bottleneck introduced by pixel observations in continuous-control locomotion.</p> <p>All results are:</p> <ul> <li>Multi-seed (seeds 0, 1, 2)</li> <li>Deterministic evaluation (mean action)</li> <li>Aggregated as mean ± standard deviation</li> <li>Fully reproducible via <code>scripts/reproduce_hopper_v4_20k.sh</code></li> </ul> <h2>Benchmark Configuration (<code>20k_eval5</code>)</h2> <ul> <li><strong>Environment:</strong> Hopper-v4 (Gymnasium + MuJoCo)</li> <li><strong>Total steps:</strong> 20,000</li> <li><strong>Eval frequency:</strong> every 2,000 steps</li> <li><strong>Eval episodes:</strong> 5 per checkpoint</li> <li><strong>PPO hyperparameters:</strong> identical across modes</li> <li><strong>Observation modes:</strong><ul> <li>State (11-dim proprioception)</li> <li>Pixels (3×64×64 RGB, CNN encoder)</li> </ul> </li> </ul> <h2>Final Deterministic Evaluation Return</h2> <p>(mean ± std across 3 seeds)</p> <p>| Mode | Return | |------|--------| | State | 204 ± 20 | | Pixels (64×64) | 80 ± 34 |</p> <p>State-based PPO achieves approximately 2.6× higher early-training return than pixel-based PPO under identical optimisation settings.</p> <p>Pixel mode uses ~710k parameters versus ~10k for state mode (~70× increase), materially altering optimisation dynamics despite identical hyperparameters.</p> <h2>Reproducibility</h2> <p>To reproduce this benchmark:</p> <pre><code>bash scripts/reproduce_hopper_v4_20k.sh</code></pre> <p>This script:</p> <ul> <li>Runs 6 training jobs (2 modes × 3 seeds)</li> <li>Aggregates results</li> <li>Regenerates plots</li> <li>Produces <code>reports/results_hopper_v4.md</code></li> </ul> <p>CI passes smoke tests and determinism checks.</p> <h2>Scope</h2> <p>This release characterises early-training behaviour only.</p> <p>20k steps is far below convergence for Hopper-v4 (typically ≥1M steps). These results describe early-training dynamics rather than asymptotic performance.</p> <p>Future releases may include:</p> <ul> <li>Frame stacking</li> <li>Higher training budgets (≥100k steps)</li> <li>Additional locomotion environments</li> </ul> <h2>Artifact Intent</h2> <p>This release is part of a broader embodied RL research portfolio focusing on:</p> <ul> <li>Safe RL (constrained optimisation)</li> <li>Vision-based control</li> <li>Robustness and deployment constraints</li> </ul> <p>If this artifact is useful for research or teaching, please consider citing the repository (Zenodo DOI forthcoming).</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18825148 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | jaintle/visual-rl-locomotion: v1.0 — 20k Multi-Seed State vs Pixel PPO Benchmark Abhinav Jain <h1>v1.0 — 20k Multi-Seed State vs Pixel PPO Benchmark</h1> <h2>Summary</h2> <p>This release marks the first reproducible benchmark of state-based vs pixel-based PPO on Hopper-v4 under controlled, identical hyperparameters.</p> <p>The goal of this repository is not to propose a new algorithm, but to quantify the representation bottleneck introduced by pixel observations in continuous-control locomotion.</p> <p>All results are:</p> <ul> <li>Multi-seed (seeds 0, 1, 2)</li> <li>Deterministic evaluation (mean action)</li> <li>Aggregated as mean ± standard deviation</li> <li>Fully reproducible via <code>scripts/reproduce_hopper_v4_20k.sh</code></li> </ul> <h2>Benchmark Configuration (<code>20k_eval5</code>)</h2> <ul> <li><strong>Environment:</strong> Hopper-v4 (Gymnasium + MuJoCo)</li> <li><strong>Total steps:</strong> 20,000</li> <li><strong>Eval frequency:</strong> every 2,000 steps</li> <li><strong>Eval episodes:</strong> 5 per checkpoint</li> <li><strong>PPO hyperparameters:</strong> identical across modes</li> <li><strong>Observation modes:</strong><ul> <li>State (11-dim proprioception)</li> <li>Pixels (3×64×64 RGB, CNN encoder)</li> </ul> </li> </ul> <h2>Final Deterministic Evaluation Return</h2> <p>(mean ± std across 3 seeds)</p> <p>| Mode | Return | |------|--------| | State | 204 ± 20 | | Pixels (64×64) | 80 ± 34 |</p> <p>State-based PPO achieves approximately 2.6× higher early-training return than pixel-based PPO under identical optimisation settings.</p> <p>Pixel mode uses ~710k parameters versus ~10k for state mode (~70× increase), materially altering optimisation dynamics despite identical hyperparameters.</p> <h2>Reproducibility</h2> <p>To reproduce this benchmark:</p> <pre><code>bash scripts/reproduce_hopper_v4_20k.sh</code></pre> <p>This script:</p> <ul> <li>Runs 6 training jobs (2 modes × 3 seeds)</li> <li>Aggregates results</li> <li>Regenerates plots</li> <li>Produces <code>reports/results_hopper_v4.md</code></li> </ul> <p>CI passes smoke tests and determinism checks.</p> <h2>Scope</h2> <p>This release characterises early-training behaviour only.</p> <p>20k steps is far below convergence for Hopper-v4 (typically ≥1M steps). These results describe early-training dynamics rather than asymptotic performance.</p> <p>Future releases may include:</p> <ul> <li>Frame stacking</li> <li>Higher training budgets (≥100k steps)</li> <li>Additional locomotion environments</li> </ul> <h2>Artifact Intent</h2> <p>This release is part of a broader embodied RL research portfolio focusing on:</p> <ul> <li>Safe RL (constrained optimisation)</li> <li>Vision-based control</li> <li>Robustness and deployment constraints</li> </ul> <p>If this artifact is useful for research or teaching, please consider citing the repository (Zenodo DOI forthcoming).</p> |
| title | jaintle/visual-rl-locomotion: v1.0 — 20k Multi-Seed State vs Pixel PPO Benchmark |
| url | https://doi.org/10.5281/zenodo.18825148 |