ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914090333503488 |
|---|---|
| author | Li, Yuhang Zhang, Chenchen Lv, Ruilin Liu, Ao Deng, Ken Zhang, Yuanxing Liu, Jiaheng Zhou, Wiggin Zhou, Bo |
| author_facet | Li, Yuhang Zhang, Chenchen Lv, Ruilin Liu, Ao Deng, Ken Zhang, Yuanxing Liu, Jiaheng Zhou, Wiggin Zhou, Bo |
| contents | While Large Language Models (LLMs) excel at algorithmic code generation, they struggle with front-end development, where correctness is judged on rendered pixels and interaction. We present ReLook, an agentic, vision-grounded reinforcement learning framework that empowers an agent to close a robust generate--diagnose--refine loop by invoking a multimodal LLM (MLLM) as a tool. During training, the agent uses the MLLM-in-the-loop both as a visual critic--scoring code with screenshots--and as a source of actionable, vision-grounded feedback; a strict zero-reward rule for invalid renders anchors renderability and prevents reward hacking. To prevent behavioral collapse, we introduce Forced Optimization, a strict acceptance rule that admits only improving revisions, yielding monotonically better trajectories. At inference, we decouple the critic and run a lightweight, critic-free self-edit cycle, keeping latency comparable to base decoding while retaining most of the gains. Across three widely used benchmarks, ReLook consistently outperforms strong baselines in vision-grounded front-end code generation, highlighting the benefits of agentic perception, visual rewards, and training-inference decoupling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_11498 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding Li, Yuhang Zhang, Chenchen Lv, Ruilin Liu, Ao Deng, Ken Zhang, Yuanxing Liu, Jiaheng Zhou, Wiggin Zhou, Bo Machine Learning Computation and Language While Large Language Models (LLMs) excel at algorithmic code generation, they struggle with front-end development, where correctness is judged on rendered pixels and interaction. We present ReLook, an agentic, vision-grounded reinforcement learning framework that empowers an agent to close a robust generate--diagnose--refine loop by invoking a multimodal LLM (MLLM) as a tool. During training, the agent uses the MLLM-in-the-loop both as a visual critic--scoring code with screenshots--and as a source of actionable, vision-grounded feedback; a strict zero-reward rule for invalid renders anchors renderability and prevents reward hacking. To prevent behavioral collapse, we introduce Forced Optimization, a strict acceptance rule that admits only improving revisions, yielding monotonically better trajectories. At inference, we decouple the critic and run a lightweight, critic-free self-edit cycle, keeping latency comparable to base decoding while retaining most of the gains. Across three widely used benchmarks, ReLook consistently outperforms strong baselines in vision-grounded front-end code generation, highlighting the benefits of agentic perception, visual rewards, and training-inference decoupling. |
| title | ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2510.11498 |