Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916070421430272 |
|---|---|
| author | Yoon, Hee Suk Yoon, Eunseop Jang, Jaehyun Eom, SooHwan Hong, Ji Woo Hasegawa-Johnson, Mark Dai, Qi Luo, Chong Yoo, Chang D. |
| author_facet | Yoon, Hee Suk Yoon, Eunseop Jang, Jaehyun Eom, SooHwan Hong, Ji Woo Hasegawa-Johnson, Mark Dai, Qi Luo, Chong Yoo, Chang D. |
| contents | While on-policy distillation offers dense supervision for training small reasoning models, its optimization dynamics in the multimodal domain remain under-explored. In this work, we challenge the standard monolithic view of Vision-Language Model (VLM) distillation by mathematically decomposing the loss into two distinct components: the language prior and visual grounding. Our analysis uncovers that gradient vectors for these components are nearly orthogonal, indicating that the objective of aligning with the teacher's language distribution is geometrically independent from the objective of matching its visual perception. Consequently, standard optimization passively follows a suboptimal compromise trajectory that implicitly balances the two objectives. Hypothesizing that visual grounding constitutes the primary bottleneck for vision-language reasoning, we introduce Visual Gradient Steering (VGS), a method that dynamically reorients the update vector to prioritize the visual subspace. Experimental results on multiple distillation settings and complex multimodal benchmarks demonstrate that VGS significantly outperforms the standard monolithic formulation of on-policy distillation, achieving superior grounding with minimal training overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_00564 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding Yoon, Hee Suk Yoon, Eunseop Jang, Jaehyun Eom, SooHwan Hong, Ji Woo Hasegawa-Johnson, Mark Dai, Qi Luo, Chong Yoo, Chang D. Computer Vision and Pattern Recognition Computation and Language While on-policy distillation offers dense supervision for training small reasoning models, its optimization dynamics in the multimodal domain remain under-explored. In this work, we challenge the standard monolithic view of Vision-Language Model (VLM) distillation by mathematically decomposing the loss into two distinct components: the language prior and visual grounding. Our analysis uncovers that gradient vectors for these components are nearly orthogonal, indicating that the objective of aligning with the teacher's language distribution is geometrically independent from the objective of matching its visual perception. Consequently, standard optimization passively follows a suboptimal compromise trajectory that implicitly balances the two objectives. Hypothesizing that visual grounding constitutes the primary bottleneck for vision-language reasoning, we introduce Visual Gradient Steering (VGS), a method that dynamically reorients the update vector to prioritize the visual subspace. Experimental results on multiple distillation settings and complex multimodal benchmarks demonstrate that VGS significantly outperforms the standard monolithic formulation of on-policy distillation, achieving superior grounding with minimal training overhead. |
| title | Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2606.00564 |