A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914317111132160 |
|---|---|
| author | Zhang, Lingzhe Fang, Liancheng Duan, Chiming He, Minghua Pan, Leyi Xiao, Pei Huang, Shiyu Zhai, Yunpeng Hu, Xuming Yu, Philip S. Liu, Aiwei |
| author_facet | Zhang, Lingzhe Fang, Liancheng Duan, Chiming He, Minghua Pan, Leyi Xiao, Pei Huang, Shiyu Zhai, Yunpeng Hu, Xuming Yu, Philip S. Liu, Aiwei |
| contents | As text generation has become a core capability of modern Large Language Models (LLMs), it underpins a wide range of downstream applications. However, most existing LLMs rely on autoregressive (AR) generation, producing one token at a time based on previously generated context-resulting in limited generation speed due to the inherently sequential nature of the process. To address this challenge, an increasing number of researchers have begun exploring parallel text generation-a broad class of techniques aimed at breaking the token-by-token generation bottleneck and improving inference efficiency. Despite growing interest, there remains a lack of comprehensive analysis on what specific techniques constitute parallel text generation and how they improve inference performance. To bridge this gap, we present a systematic survey of parallel text generation methods. We categorize existing approaches into AR-based and Non-AR-based paradigms, and provide a detailed examination of the core techniques within each category. Following this taxonomy, we assess their theoretical trade-offs in terms of speed, quality, and efficiency, and examine their potential for combination and comparison with alternative acceleration strategies. Finally, based on our findings, we highlight recent advancements, identify open challenges, and outline promising directions for future research in parallel text generation. We have also created a GitHub repository for indexing relevant papers and open resources available at https://github.com/zhanglingzhe0820/Awesome-Parallel-Text-Generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08712 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models Zhang, Lingzhe Fang, Liancheng Duan, Chiming He, Minghua Pan, Leyi Xiao, Pei Huang, Shiyu Zhai, Yunpeng Hu, Xuming Yu, Philip S. Liu, Aiwei Computation and Language Artificial Intelligence Distributed, Parallel, and Cluster Computing 68T50 I.2.7 As text generation has become a core capability of modern Large Language Models (LLMs), it underpins a wide range of downstream applications. However, most existing LLMs rely on autoregressive (AR) generation, producing one token at a time based on previously generated context-resulting in limited generation speed due to the inherently sequential nature of the process. To address this challenge, an increasing number of researchers have begun exploring parallel text generation-a broad class of techniques aimed at breaking the token-by-token generation bottleneck and improving inference efficiency. Despite growing interest, there remains a lack of comprehensive analysis on what specific techniques constitute parallel text generation and how they improve inference performance. To bridge this gap, we present a systematic survey of parallel text generation methods. We categorize existing approaches into AR-based and Non-AR-based paradigms, and provide a detailed examination of the core techniques within each category. Following this taxonomy, we assess their theoretical trade-offs in terms of speed, quality, and efficiency, and examine their potential for combination and comparison with alternative acceleration strategies. Finally, based on our findings, we highlight recent advancements, identify open challenges, and outline promising directions for future research in parallel text generation. We have also created a GitHub repository for indexing relevant papers and open resources available at https://github.com/zhanglingzhe0820/Awesome-Parallel-Text-Generation. |
| title | A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models |
| topic | Computation and Language Artificial Intelligence Distributed, Parallel, and Cluster Computing 68T50 I.2.7 |
| url | https://arxiv.org/abs/2508.08712 |