Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910055025082368 |
|---|---|
| author | Wu, Jingxuan Wan, Zhenglin Yu, Xingrui Yang, Yuzhe Huang, Yiqiao Tsang, Ivor You, Yang |
| author_facet | Wu, Jingxuan Wan, Zhenglin Yu, Xingrui Yang, Yuzhe Huang, Yiqiao Tsang, Ivor You, Yang |
| contents | Diffusion language models (Diffusion-LMs) introduce an explicit temporal dimension into text generation, yet how this structure can be leveraged to control generation diversity for exploring multiple valid semantic or reasoning paths remains underexplored. In this paper, we show that Diffusion-LMs, like diffusion models in image generation, exhibit a temporal division of labor: early denoising steps largely determine the global semantic structure, while later steps focus on local lexical refinement. Building on this insight, we propose Time-Annealed Perturbation Sampling (TAPS), a training-free inference strategy that encourages semantic branching early in the diffusion process while progressively reducing perturbations to preserve fluency and instruction adherence. TAPS is compatible with both non-autoregressive and semi-autoregressive Diffusion backbones, demonstrated on LLaDA and TraDo in our paper, and consistently improves output diversity across creative writing and reasoning benchmarks without compromising generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22629 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models Wu, Jingxuan Wan, Zhenglin Yu, Xingrui Yang, Yuzhe Huang, Yiqiao Tsang, Ivor You, Yang Computation and Language Artificial Intelligence Diffusion language models (Diffusion-LMs) introduce an explicit temporal dimension into text generation, yet how this structure can be leveraged to control generation diversity for exploring multiple valid semantic or reasoning paths remains underexplored. In this paper, we show that Diffusion-LMs, like diffusion models in image generation, exhibit a temporal division of labor: early denoising steps largely determine the global semantic structure, while later steps focus on local lexical refinement. Building on this insight, we propose Time-Annealed Perturbation Sampling (TAPS), a training-free inference strategy that encourages semantic branching early in the diffusion process while progressively reducing perturbations to preserve fluency and instruction adherence. TAPS is compatible with both non-autoregressive and semi-autoregressive Diffusion backbones, demonstrated on LLaDA and TraDo in our paper, and consistently improves output diversity across creative writing and reasoning benchmarks without compromising generation quality. |
| title | Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.22629 |