D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914409409937408 |
|---|---|
| author | Lys, Jonathan Gripon, Vincent Pasdeloup, Bastien Marmoret, Axel Mauch, Lukas Cardinaux, Fabien Hacene, Ghouthi Boukli |
| author_facet | Lys, Jonathan Gripon, Vincent Pasdeloup, Bastien Marmoret, Axel Mauch, Lukas Cardinaux, Fabien Hacene, Ghouthi Boukli |
| contents | Discrete diffusion models are promising alternatives to autoregressive approaches for text generation, yet their decoding methods remain under-studied. Standard decoding methods for autoregressive models, such as beam search, do not directly apply to iterative denoising, and existing diffusion decoding techniques provide limited control over in-batch diversity. To bridge this gap, we introduce a generalized beam-search framework for discrete diffusion that generates candidates in parallel and supports modular beam-selection objectives. As a diversity-focused instantiation, we propose D5P4, which formulates the selection step as MAP inference over a Determinantal Point Process. Leveraging a scalable greedy solver, D5P4 maintains multi-GPU compatibility and enables an explicit trade-off between model probability and target diversity with near-zero compute overhead. Experiments on free-form generation and question answering demonstrate that D5P4 improves diversity over strong baselines while maintaining competitive generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19146 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding Lys, Jonathan Gripon, Vincent Pasdeloup, Bastien Marmoret, Axel Mauch, Lukas Cardinaux, Fabien Hacene, Ghouthi Boukli Artificial Intelligence Machine Learning Discrete diffusion models are promising alternatives to autoregressive approaches for text generation, yet their decoding methods remain under-studied. Standard decoding methods for autoregressive models, such as beam search, do not directly apply to iterative denoising, and existing diffusion decoding techniques provide limited control over in-batch diversity. To bridge this gap, we introduce a generalized beam-search framework for discrete diffusion that generates candidates in parallel and supports modular beam-selection objectives. As a diversity-focused instantiation, we propose D5P4, which formulates the selection step as MAP inference over a Determinantal Point Process. Leveraging a scalable greedy solver, D5P4 maintains multi-GPU compatibility and enables an explicit trade-off between model probability and target diversity with near-zero compute overhead. Experiments on free-form generation and question answering demonstrate that D5P4 improves diversity over strong baselines while maintaining competitive generation quality. |
| title | D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.19146 |