Noise Scheduling as Information-Guided Allocation in Diffusion Training
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916054706421760 |
|---|---|
| author | Raya, Gabriel Nguyen, Bac Batzolis, Georgios Takida, Yuhta Stancevic, Dejan Murata, Naoki Lai, Chieh-Hsin Mitsufuji, Yuki Ambrogioni, Luca |
| author_facet | Raya, Gabriel Nguyen, Bac Batzolis, Georgios Takida, Yuhta Stancevic, Dejan Murata, Naoki Lai, Chieh-Hsin Mitsufuji, Yuki Ambrogioni, Luca |
| contents | We introduce InfoNoise, an online adaptive noise schedule for diffusion training that reallocates optimization effort toward noise levels where denoising is most informative. Together with loss weighting, a noise schedule induces an effective allocation across denoising problems, often fixed before informative noise levels are known. InfoNoise makes this allocation data-adaptive by estimating a conditional-entropy-rate profile from denoising losses during training, without auxiliary models or offline search. Through I--MMSE, this profile identifies where noisy observations rapidly reduce uncertainty about the clean sample and guides adaptation of the training noise distribution. It changes only this distribution, keeping the objective, weighting, and parameterization fixed. On image benchmarks, where schedules have been extensively tuned, InfoNoise matches or slightly exceeds strong baselines and can reach the same quality with fewer updates. On representation, sequence, and modality shifts, including DNA and language generation, InfoNoise improves over fixed and adaptive baselines and reaches target quality with up to $3\times$ less training compute. These results establish the conditional-entropy-rate profile as the data-dependent target for noise schedule design and make online adaptation a practical alternative to manual schedule search. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18647 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Noise Scheduling as Information-Guided Allocation in Diffusion Training Raya, Gabriel Nguyen, Bac Batzolis, Georgios Takida, Yuhta Stancevic, Dejan Murata, Naoki Lai, Chieh-Hsin Mitsufuji, Yuki Ambrogioni, Luca Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Information Theory We introduce InfoNoise, an online adaptive noise schedule for diffusion training that reallocates optimization effort toward noise levels where denoising is most informative. Together with loss weighting, a noise schedule induces an effective allocation across denoising problems, often fixed before informative noise levels are known. InfoNoise makes this allocation data-adaptive by estimating a conditional-entropy-rate profile from denoising losses during training, without auxiliary models or offline search. Through I--MMSE, this profile identifies where noisy observations rapidly reduce uncertainty about the clean sample and guides adaptation of the training noise distribution. It changes only this distribution, keeping the objective, weighting, and parameterization fixed. On image benchmarks, where schedules have been extensively tuned, InfoNoise matches or slightly exceeds strong baselines and can reach the same quality with fewer updates. On representation, sequence, and modality shifts, including DNA and language generation, InfoNoise improves over fixed and adaptive baselines and reaches target quality with up to $3\times$ less training compute. These results establish the conditional-entropy-rate profile as the data-dependent target for noise schedule design and make online adaptation a practical alternative to manual schedule search. |
| title | Noise Scheduling as Information-Guided Allocation in Diffusion Training |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Information Theory |
| url | https://arxiv.org/abs/2602.18647 |