d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911449143574528 |
|---|---|
| author | Jiang, Yuchu Cai, Yue Luo, Xiangzhong Fu, Jiale Wang, Jiarui Liu, Chonghan Yang, Xu |
| author_facet | Jiang, Yuchu Cai, Yue Luo, Xiangzhong Fu, Jiale Wang, Jiarui Liu, Chonghan Yang, Xu |
| contents | Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code is available at https://github.com/Kamichanw/d2Cache. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23094 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching Jiang, Yuchu Cai, Yue Luo, Xiangzhong Fu, Jiale Wang, Jiarui Liu, Chonghan Yang, Xu Computation and Language Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code is available at https://github.com/Kamichanw/d2Cache. |
| title | d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.23094 |