Variance Reduction for Expectations with Diffusion Teachers
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918517334343680 |
|---|---|
| author | Bettencourt, Jesse Wu, Xindi Atzmon, Matan Lucas, James Lorraine, Jonathan |
| author_facet | Bettencourt, Jesse Wu, Xindi Atzmon, Matan Lucas, James Lorraine, Jonathan |
| contents | Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most from amortized reuse; ~25% additional from IS+stratification) without changing the objective; in single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_21489 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Variance Reduction for Expectations with Diffusion Teachers Bettencourt, Jesse Wu, Xindi Atzmon, Matan Lucas, James Lorraine, Jonathan Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Computation 65C05 I.2.6; G.3 Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most from amortized reuse; ~25% additional from IS+stratification) without changing the objective; in single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck. |
| title | Variance Reduction for Expectations with Diffusion Teachers |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Computation 65C05 I.2.6; G.3 |
| url | https://arxiv.org/abs/2605.21489 |