$π$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910045802856448 |
|---|---|
| author | Wang, Siting Wang, Xiaofeng Zhu, Zheng Pei, Minnan Cui, Xinyu Deng, Cheng Zhao, Jian Huang, Guan Zhang, Haifeng Wang, Jun |
| author_facet | Wang, Siting Wang, Xiaofeng Zhu, Zheng Pei, Minnan Cui, Xinyu Deng, Cheng Zhao, Jian Huang, Guan Zhang, Haifeng Wang, Jun |
| contents | Flow-based vision-language-action (VLA) models excel in embodied control but suffer from intractable likelihoods during multi-step sampling, hindering online reinforcement learning. We propose \textbf{\textit{$\boldsymbolπ$-StepNFT}} (Step-wise Negative-aware Fine-Tuning), a critic-and-likelihood-free framework that requires only a single forward pass per optimization step and eliminates auxiliary value networks. We identify that wider exploration spaces necessitate finer-grained, step-wise guidance for alignment. Empirically, $π$-StepNFT unlocks latent potential on LIBERO with competitive few-shot robustness. Moreover, it achieves superior generalization on ManiSkill, outperforming value-based baselines in OOD scenarios by preventing overfitting to multimodal features. This property offers a scalable solution promising for complex real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02083 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | $π$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs Wang, Siting Wang, Xiaofeng Zhu, Zheng Pei, Minnan Cui, Xinyu Deng, Cheng Zhao, Jian Huang, Guan Zhang, Haifeng Wang, Jun Robotics Computer Vision and Pattern Recognition Flow-based vision-language-action (VLA) models excel in embodied control but suffer from intractable likelihoods during multi-step sampling, hindering online reinforcement learning. We propose \textbf{\textit{$\boldsymbolπ$-StepNFT}} (Step-wise Negative-aware Fine-Tuning), a critic-and-likelihood-free framework that requires only a single forward pass per optimization step and eliminates auxiliary value networks. We identify that wider exploration spaces necessitate finer-grained, step-wise guidance for alignment. Empirically, $π$-StepNFT unlocks latent potential on LIBERO with competitive few-shot robustness. Moreover, it achieves superior generalization on ManiSkill, outperforming value-based baselines in OOD scenarios by preventing overfitting to multimodal features. This property offers a scalable solution promising for complex real-world applications. |
| title | $π$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.02083 |