OCALM: Object-Centric Assessment with Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929397928296448 |
|---|---|
| author | Kaufmann, Timo Blüml, Jannis Wüst, Antonia Delfosse, Quentin Kersting, Kristian Hüllermeier, Eyke |
| author_facet | Kaufmann, Timo Blüml, Jannis Wüst, Antonia Delfosse, Quentin Kersting, Kristian Hüllermeier, Eyke |
| contents | Properly defining a reward signal to efficiently train a reinforcement learning (RL) agent is a challenging task. Designing balanced objective functions from which a desired behavior can emerge requires expert knowledge, especially for complex environments. Learning rewards from human feedback or using large language models (LLMs) to directly provide rewards are promising alternatives, allowing non-experts to specify goals for the agent. However, black-box reward models make it difficult to debug the reward. In this work, we propose Object-Centric Assessment with Language Models (OCALM) to derive inherently interpretable reward functions for RL agents from natural language task descriptions. OCALM uses the extensive world-knowledge of LLMs while leveraging the object-centric nature common to many environments to derive reward functions focused on relational concepts, providing RL agents with the ability to derive policies from task descriptions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_16748 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OCALM: Object-Centric Assessment with Language Models Kaufmann, Timo Blüml, Jannis Wüst, Antonia Delfosse, Quentin Kersting, Kristian Hüllermeier, Eyke Machine Learning Computation and Language Properly defining a reward signal to efficiently train a reinforcement learning (RL) agent is a challenging task. Designing balanced objective functions from which a desired behavior can emerge requires expert knowledge, especially for complex environments. Learning rewards from human feedback or using large language models (LLMs) to directly provide rewards are promising alternatives, allowing non-experts to specify goals for the agent. However, black-box reward models make it difficult to debug the reward. In this work, we propose Object-Centric Assessment with Language Models (OCALM) to derive inherently interpretable reward functions for RL agents from natural language task descriptions. OCALM uses the extensive world-knowledge of LLMs while leveraging the object-centric nature common to many environments to derive reward functions focused on relational concepts, providing RL agents with the ability to derive policies from task descriptions. |
| title | OCALM: Object-Centric Assessment with Language Models |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2406.16748 |