Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.20675 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909974912827392 |
|---|---|
| author | Roy, Simon Barbeau, Samuel Beltrame, Giovanni Desrosiers, Christian Thome, Nicolas |
| author_facet | Roy, Simon Barbeau, Samuel Beltrame, Giovanni Desrosiers, Christian Thome, Nicolas |
| contents | Learning generalizable reward functions is a core challenge in embodied intelligence. Recent work leverages contrastive vision language models (VLMs) to obtain dense, domain-agnostic rewards without human supervision. These methods adapt VLMs into reward models through increasingly complex learning objectives, yet meaningful comparison remains difficult due to differences in training data, architectures, and evaluation settings. In this work, we isolate the impact of the learning objective by evaluating recent VLM-based reward models under a unified framework with identical backbones, finetuning data, and evaluation environments. Using Meta-World tasks, we assess modeling accuracy by measuring consistency with ground truth reward and correlation with expert progress. Remarkably, we show that a simple triplet loss outperforms state-of-the-art methods, suggesting that much of the improvements in recent approaches could be attributed to differences in data and architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20675 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Revisiting the Learning Objectives of Vision-Language Reward Models Roy, Simon Barbeau, Samuel Beltrame, Giovanni Desrosiers, Christian Thome, Nicolas Machine Learning Artificial Intelligence Learning generalizable reward functions is a core challenge in embodied intelligence. Recent work leverages contrastive vision language models (VLMs) to obtain dense, domain-agnostic rewards without human supervision. These methods adapt VLMs into reward models through increasingly complex learning objectives, yet meaningful comparison remains difficult due to differences in training data, architectures, and evaluation settings. In this work, we isolate the impact of the learning objective by evaluating recent VLM-based reward models under a unified framework with identical backbones, finetuning data, and evaluation environments. Using Meta-World tasks, we assess modeling accuracy by measuring consistency with ground truth reward and correlation with expert progress. Remarkably, we show that a simple triplet loss outperforms state-of-the-art methods, suggesting that much of the improvements in recent approaches could be attributed to differences in data and architectures. |
| title | Revisiting the Learning Objectives of Vision-Language Reward Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.20675 |