O3D: Offline Data-driven Discovery and Distillation for Sequential Decision-Making with Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910344213954560 |
|---|---|
| author | Xiao, Yuchen Sun, Yanchao Xu, Mengda Madhushani, Udari Vann, Jared Garg, Deepeka Ganesh, Sumitra |
| author_facet | Xiao, Yuchen Sun, Yanchao Xu, Mengda Madhushani, Udari Vann, Jared Garg, Deepeka Ganesh, Sumitra |
| contents | Recent advancements in large language models (LLMs) have exhibited promising performance in solving sequential decision-making problems. By imitating few-shot examples provided in the prompts (i.e., in-context learning), an LLM agent can interact with an external environment and complete given tasks without additional training. However, such few-shot examples are often insufficient to generate high-quality solutions for complex and long-horizon tasks, while the limited context length cannot consume larger-scale demonstrations with long interaction horizons. To this end, we propose an offline learning framework that utilizes offline data at scale (e.g, logs of human interactions) to improve LLM-powered policies without finetuning. The proposed method O3D (Offline Data-driven Discovery and Distillation) automatically discovers reusable skills and distills generalizable knowledge across multiple tasks based on offline interaction data, advancing the capability of solving downstream tasks. Empirical results under two interactive decision-making benchmarks (ALFWorld and WebShop) verify that O3D can notably enhance the decision-making capabilities of LLMs through the offline discovery and distillation process, and consistently outperform baselines across various LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_14403 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | O3D: Offline Data-driven Discovery and Distillation for Sequential Decision-Making with Large Language Models Xiao, Yuchen Sun, Yanchao Xu, Mengda Madhushani, Udari Vann, Jared Garg, Deepeka Ganesh, Sumitra Artificial Intelligence Computation and Language Recent advancements in large language models (LLMs) have exhibited promising performance in solving sequential decision-making problems. By imitating few-shot examples provided in the prompts (i.e., in-context learning), an LLM agent can interact with an external environment and complete given tasks without additional training. However, such few-shot examples are often insufficient to generate high-quality solutions for complex and long-horizon tasks, while the limited context length cannot consume larger-scale demonstrations with long interaction horizons. To this end, we propose an offline learning framework that utilizes offline data at scale (e.g, logs of human interactions) to improve LLM-powered policies without finetuning. The proposed method O3D (Offline Data-driven Discovery and Distillation) automatically discovers reusable skills and distills generalizable knowledge across multiple tasks based on offline interaction data, advancing the capability of solving downstream tasks. Empirical results under two interactive decision-making benchmarks (ALFWorld and WebShop) verify that O3D can notably enhance the decision-making capabilities of LLMs through the offline discovery and distillation process, and consistently outperform baselines across various LLMs. |
| title | O3D: Offline Data-driven Discovery and Distillation for Sequential Decision-Making with Large Language Models |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2310.14403 |