Embedding-Aligned Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916456517599232 |
|---|---|
| author | Tennenholtz, Guy Chow, Yinlam Hsu, Chih-Wei Shani, Lior Liang, Ethan Boutilier, Craig |
| author_facet | Tennenholtz, Guy Chow, Yinlam Hsu, Chih-Wei Shani, Lior Liang, Ethan Boutilier, Craig |
| contents | We propose a novel approach for training large language models (LLMs) to adhere to objectives defined within a latent embedding space. Our method leverages reinforcement learning (RL), treating a pre-trained LLM as an environment. Our embedding-aligned guided language (EAGLE) agent is trained to iteratively steer the LLM's generation towards optimal regions of the latent embedding space, w.r.t. some predefined criterion. We demonstrate the effectiveness of the EAGLE agent using the MovieLens 25M and Amazon Review datasets to surface content gaps that satisfy latent user demand. We also demonstrate the benefit of using an optimal design of a state-dependent action set to improve EAGLE's efficiency. Our work paves the way for controlled and grounded text generation using LLMs, ensuring consistency with domain-specific knowledge and data representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00024 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Embedding-Aligned Language Models Tennenholtz, Guy Chow, Yinlam Hsu, Chih-Wei Shani, Lior Liang, Ethan Boutilier, Craig Computation and Language Artificial Intelligence Emerging Technologies Machine Learning We propose a novel approach for training large language models (LLMs) to adhere to objectives defined within a latent embedding space. Our method leverages reinforcement learning (RL), treating a pre-trained LLM as an environment. Our embedding-aligned guided language (EAGLE) agent is trained to iteratively steer the LLM's generation towards optimal regions of the latent embedding space, w.r.t. some predefined criterion. We demonstrate the effectiveness of the EAGLE agent using the MovieLens 25M and Amazon Review datasets to surface content gaps that satisfy latent user demand. We also demonstrate the benefit of using an optimal design of a state-dependent action set to improve EAGLE's efficiency. Our work paves the way for controlled and grounded text generation using LLMs, ensuring consistency with domain-specific knowledge and data representations. |
| title | Embedding-Aligned Language Models |
| topic | Computation and Language Artificial Intelligence Emerging Technologies Machine Learning |
| url | https://arxiv.org/abs/2406.00024 |