DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915767388209152 |
|---|---|
| author | Wu, Zirui Zheng, Lin Xie, Zhihui Ye, Jiacheng Gao, Jiahui Gong, Shansan Feng, Yansong Li, Zhenguo Bi, Wei Zhou, Guorui Kong, Lingpeng |
| author_facet | Wu, Zirui Zheng, Lin Xie, Zhihui Ye, Jiacheng Gao, Jiahui Gong, Shansan Feng, Yansong Li, Zhenguo Bi, Wei Zhou, Guorui Kong, Lingpeng |
| contents | Diffusion Language Models (DLMs) present a compelling alternative to autoregressive models, offering flexible, any-order infilling without specialized prompting design. However, their practical utility is blocked by a critical limitation: the requirement of a fixed-length masked sequence for generation. This constraint severely degrades code infilling performance when the predefined mask size mismatches the ideal completion length. To address this, we propose DreamOn, a novel diffusion framework that enables dynamic, variable-length generation. DreamOn augments the diffusion process with two length control states, allowing the model to autonomously expand or contract the output length based solely on its own predictions. We integrate this mechanism into existing DLMs with minimal modifications to the training objective and no architectural changes. Built upon Dream-Coder-7B and DiffuCoder-7B, DreamOn achieves infilling performance on par with state-of-the-art autoregressive models on HumanEval-Infilling and SantaCoder-FIM and matches oracle performance achieved with ground-truth length. Our work removes a fundamental barrier to the practical deployment of DLMs, significantly advancing their flexibility and applicability for variable-length generation. Our code is available at https://github.com/DreamLM/DreamOn. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01326 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas Wu, Zirui Zheng, Lin Xie, Zhihui Ye, Jiacheng Gao, Jiahui Gong, Shansan Feng, Yansong Li, Zhenguo Bi, Wei Zhou, Guorui Kong, Lingpeng Computation and Language Diffusion Language Models (DLMs) present a compelling alternative to autoregressive models, offering flexible, any-order infilling without specialized prompting design. However, their practical utility is blocked by a critical limitation: the requirement of a fixed-length masked sequence for generation. This constraint severely degrades code infilling performance when the predefined mask size mismatches the ideal completion length. To address this, we propose DreamOn, a novel diffusion framework that enables dynamic, variable-length generation. DreamOn augments the diffusion process with two length control states, allowing the model to autonomously expand or contract the output length based solely on its own predictions. We integrate this mechanism into existing DLMs with minimal modifications to the training objective and no architectural changes. Built upon Dream-Coder-7B and DiffuCoder-7B, DreamOn achieves infilling performance on par with state-of-the-art autoregressive models on HumanEval-Infilling and SantaCoder-FIM and matches oracle performance achieved with ground-truth length. Our work removes a fundamental barrier to the practical deployment of DLMs, significantly advancing their flexibility and applicability for variable-length generation. Our code is available at https://github.com/DreamLM/DreamOn. |
| title | DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.01326 |