Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909556592869376 |
|---|---|
| author | Chae, Hyungjoo Kim, Namyoung Ong, Kai Tzu-iunn Gwak, Minju Song, Gwanwoo Kim, Jihoon Kim, Sunghwan Lee, Dongha Yeo, Jinyoung |
| author_facet | Chae, Hyungjoo Kim, Namyoung Ong, Kai Tzu-iunn Gwak, Minju Song, Gwanwoo Kim, Jihoon Kim, Sunghwan Lee, Dongha Yeo, Jinyoung |
| contents | Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model". Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet, etc.). Then, we present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13232 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation Chae, Hyungjoo Kim, Namyoung Ong, Kai Tzu-iunn Gwak, Minju Song, Gwanwoo Kim, Jihoon Kim, Sunghwan Lee, Dongha Yeo, Jinyoung Computation and Language Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model". Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet, etc.). Then, we present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents. |
| title | Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.13232 |