Dynamic Mixture-of-Experts for Visual Autoregressive Model
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918387556286464 |
|---|---|
| author | Vincenti, Jort Jazbec, Metod Xia, Guoxuan |
| author_facet | Vincenti, Jort Jazbec, Metod Xia, Guoxuan |
| contents | Visual Autoregressive Models (VAR) offer efficient and high-quality image generation but suffer from computational redundancy due to repeated Transformer calls at increasing resolutions. We introduce a dynamic Mixture-of-Experts router integrated into VAR. The new architecture allows to trade compute for quality through scale-aware thresholding. This thresholding strategy balances expert selection based on token complexity and resolution, without requiring additional training. As a result, we achieve 20% fewer FLOPs, 11% faster inference and match the image quality achieved by the dense baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08629 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Mixture-of-Experts for Visual Autoregressive Model Vincenti, Jort Jazbec, Metod Xia, Guoxuan Computer Vision and Pattern Recognition Visual Autoregressive Models (VAR) offer efficient and high-quality image generation but suffer from computational redundancy due to repeated Transformer calls at increasing resolutions. We introduce a dynamic Mixture-of-Experts router integrated into VAR. The new architecture allows to trade compute for quality through scale-aware thresholding. This thresholding strategy balances expert selection based on token complexity and resolution, without requiring additional training. As a result, we achieve 20% fewer FLOPs, 11% faster inference and match the image quality achieved by the dense baseline. |
| title | Dynamic Mixture-of-Experts for Visual Autoregressive Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.08629 |