RewardHarness: Self-Evolving Agentic Post-Training
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917476533534720 |
|---|---|
| author | Zhang, Yuxuan Du, Penghui Li, Bo Wei, Cong Miao, Junwen Zhang, Huaisong Cai, Songcheng Wang, Yubo Jiang, Dongfu Zhang, Yuyu Nie, Ping Chen, Wenhu Yu, Changqian Allen, Kelsey R. |
| author_facet | Zhang, Yuxuan Du, Penghui Li, Bo Wei, Cong Miao, Junwen Zhang, Huaisong Cai, Songcheng Wang, Yubo Jiang, Dongfu Zhang, Yuyu Nie, Ping Chen, Wenhu Yu, Changqian Allen, Kelsey R. |
| contents | Evaluating instruction-guided image edits requires rewards that reflect subtle human preferences, yet current reward models typically depend on large-scale preference annotation and additional model training. This creates a data-efficiency gap: humans can often infer the target evaluation criteria from only a few examples, while models are usually trained on hundreds of thousands of comparisons. We present RewardHarness, a self-evolving agentic reward framework that reframes reward modeling as context evolution rather than weight optimization. Instead of learning from large-scale annotations, RewardHarness aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations. Given a source image, candidate edited images, and an editing instruction, an Orchestrator selects the most relevant subset of tools and skills from the maintained library, and a frozen Sub-Agent uses them to construct a reasoning chain that produces a preference judgment. By comparing predicted judgments with ground-truth preferences and analyzing successes and failures in the reasoning process, the Orchestrator automatically refines its library of tools and skills without additional human annotation. Using only 0.05% of the EditReward preference data, RewardHarness achieves 47.4% average accuracy on image-editing evaluation benchmarks, surpassing GPT-5 by 5.3 points. When used as a reward signal for GRPO fine-tuning, RL-tuned models achieve 3.52 on ImgEdit-Bench. Project page: https://rewardharness.com. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08703 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RewardHarness: Self-Evolving Agentic Post-Training Zhang, Yuxuan Du, Penghui Li, Bo Wei, Cong Miao, Junwen Zhang, Huaisong Cai, Songcheng Wang, Yubo Jiang, Dongfu Zhang, Yuyu Nie, Ping Chen, Wenhu Yu, Changqian Allen, Kelsey R. Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning Evaluating instruction-guided image edits requires rewards that reflect subtle human preferences, yet current reward models typically depend on large-scale preference annotation and additional model training. This creates a data-efficiency gap: humans can often infer the target evaluation criteria from only a few examples, while models are usually trained on hundreds of thousands of comparisons. We present RewardHarness, a self-evolving agentic reward framework that reframes reward modeling as context evolution rather than weight optimization. Instead of learning from large-scale annotations, RewardHarness aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations. Given a source image, candidate edited images, and an editing instruction, an Orchestrator selects the most relevant subset of tools and skills from the maintained library, and a frozen Sub-Agent uses them to construct a reasoning chain that produces a preference judgment. By comparing predicted judgments with ground-truth preferences and analyzing successes and failures in the reasoning process, the Orchestrator automatically refines its library of tools and skills without additional human annotation. Using only 0.05% of the EditReward preference data, RewardHarness achieves 47.4% average accuracy on image-editing evaluation benchmarks, surpassing GPT-5 by 5.3 points. When used as a reward signal for GRPO fine-tuning, RL-tuned models achieve 3.52 on ImgEdit-Bench. Project page: https://rewardharness.com. |
| title | RewardHarness: Self-Evolving Agentic Post-Training |
| topic | Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2605.08703 |