SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915955302465536 |
|---|---|
| author | Wang, Jichao Bian, Liuyang Zhou, Yufeng Xiao, Han Pan, Yue Wang, Guozhi Wang, Hao Wang, Zhaoxiong Wen, Yafei Chen, Xiaoxin Ren, Shuai Zeng, Lingfang |
| author_facet | Wang, Jichao Bian, Liuyang Zhou, Yufeng Xiao, Han Pan, Yue Wang, Guozhi Wang, Hao Wang, Zhaoxiong Wen, Yafei Chen, Xiaoxin Ren, Shuai Zeng, Lingfang |
| contents | As Multimodal Large Language Models (MLLMs) mature, GUI agents are evolving from static interactions to complex navigation. While Reinforcement Learning (RL) has emerged as a promising paradigm for training MLLM agents on dynamic GUI tasks, its effective application faces a dilemma. Standard Offline RL often relies on static step-level data, neglecting global trajectory semantics such as task completion and execution quality. Conversely, Online RL captures the long-term dynamics but suffers from high interaction costs and potential environmental instability. To bridge this gap, we propose SOLAR-RL (Semi-Online Long-horizon Assignment Reinforcement Learning). Instead of relying solely on expensive online interactions, our framework integrates global trajectory insights directly into the offline learning process. Specifically, we reconstruct diverse rollout candidates from static data, detect the first failure point using per-step validity signals, and retroactively assign dense step-level rewards with target-aligned shaping to reflect trajectory-level execution quality, effectively simulating online feedback without interaction costs. Extensive experiments demonstrate that SOLAR-RL significantly improves long-horizon task completion rates and robustness compared to strong baselines, offering a sample-efficient solution for autonomous GUI navigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22558 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning Wang, Jichao Bian, Liuyang Zhou, Yufeng Xiao, Han Pan, Yue Wang, Guozhi Wang, Hao Wang, Zhaoxiong Wen, Yafei Chen, Xiaoxin Ren, Shuai Zeng, Lingfang Machine Learning Artificial Intelligence As Multimodal Large Language Models (MLLMs) mature, GUI agents are evolving from static interactions to complex navigation. While Reinforcement Learning (RL) has emerged as a promising paradigm for training MLLM agents on dynamic GUI tasks, its effective application faces a dilemma. Standard Offline RL often relies on static step-level data, neglecting global trajectory semantics such as task completion and execution quality. Conversely, Online RL captures the long-term dynamics but suffers from high interaction costs and potential environmental instability. To bridge this gap, we propose SOLAR-RL (Semi-Online Long-horizon Assignment Reinforcement Learning). Instead of relying solely on expensive online interactions, our framework integrates global trajectory insights directly into the offline learning process. Specifically, we reconstruct diverse rollout candidates from static data, detect the first failure point using per-step validity signals, and retroactively assign dense step-level rewards with target-aligned shaping to reflect trajectory-level execution quality, effectively simulating online feedback without interaction costs. Extensive experiments demonstrate that SOLAR-RL significantly improves long-horizon task completion rates and robustness compared to strong baselines, offering a sample-efficient solution for autonomous GUI navigation. |
| title | SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.22558 |