Visual Self-Refinement for Autoregressive Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911187896107008 |
|---|---|
| author | Wang, Jiamian Zhou, Ziqi Mummadi, Chaithanya Kumar Dianat, Sohail Rabbani, Majid Rao, Raghuveer Qiu, Chen Tao, Zhiqiang |
| author_facet | Wang, Jiamian Zhou, Ziqi Mummadi, Chaithanya Kumar Dianat, Sohail Rabbani, Majid Rao, Raghuveer Qiu, Chen Tao, Zhiqiang |
| contents | Autoregressive models excel in sequential modeling and have proven to be effective for vision-language data. However, the spatial nature of visual signals conflicts with the sequential dependencies of next-token prediction, leading to suboptimal results. This work proposes a plug-and-play refinement module to enhance the complex spatial correspondence modeling within the generated visual sequence. This module operates as a post-pretraining step to jointly refine all generated tokens of autoregressive model, enhancing vision-language modeling under a shared sequential prediction framework. By leveraging global context and relationship across the tokens, our method mitigates the error accumulation issue within the sequential generation. Experiments demonstrate that the proposed method improves the generation quality, enhancing the model's ability to produce semantically consistent results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00993 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Visual Self-Refinement for Autoregressive Models Wang, Jiamian Zhou, Ziqi Mummadi, Chaithanya Kumar Dianat, Sohail Rabbani, Majid Rao, Raghuveer Qiu, Chen Tao, Zhiqiang Computer Vision and Pattern Recognition Autoregressive models excel in sequential modeling and have proven to be effective for vision-language data. However, the spatial nature of visual signals conflicts with the sequential dependencies of next-token prediction, leading to suboptimal results. This work proposes a plug-and-play refinement module to enhance the complex spatial correspondence modeling within the generated visual sequence. This module operates as a post-pretraining step to jointly refine all generated tokens of autoregressive model, enhancing vision-language modeling under a shared sequential prediction framework. By leveraging global context and relationship across the tokens, our method mitigates the error accumulation issue within the sequential generation. Experiments demonstrate that the proposed method improves the generation quality, enhancing the model's ability to produce semantically consistent results. |
| title | Visual Self-Refinement for Autoregressive Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.00993 |