Doppelgänger's Watch: A Split Objective Approach to Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929493880340480 |
|---|---|
| author | Ghasemlou, Shervin Katiyar, Ashish Saraf, Aparajita Moon, Seungwhan Pujari, Mangesh Donmez, Pinar Damavandi, Babak Kumar, Anuj |
| author_facet | Ghasemlou, Shervin Katiyar, Ashish Saraf, Aparajita Moon, Seungwhan Pujari, Mangesh Donmez, Pinar Damavandi, Babak Kumar, Anuj |
| contents | In this paper, we investigate the problem of "generation supervision" in large language models, and present a novel bicameral architecture to separate supervision signals from their core capability, helpfulness. Doppelgänger, a new module parallel to the underlying language model, supervises the generation of each token, and learns to concurrently predict the supervision score(s) of the sequences up to and including each token. In this work, we present the theoretical findings, and leave the report on experimental results to a forthcoming publication. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_06107 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Doppelgänger's Watch: A Split Objective Approach to Large Language Models Ghasemlou, Shervin Katiyar, Ashish Saraf, Aparajita Moon, Seungwhan Pujari, Mangesh Donmez, Pinar Damavandi, Babak Kumar, Anuj Computation and Language Artificial Intelligence In this paper, we investigate the problem of "generation supervision" in large language models, and present a novel bicameral architecture to separate supervision signals from their core capability, helpfulness. Doppelgänger, a new module parallel to the underlying language model, supervises the generation of each token, and learns to concurrently predict the supervision score(s) of the sequences up to and including each token. In this work, we present the theoretical findings, and leave the report on experimental results to a forthcoming publication. |
| title | Doppelgänger's Watch: A Split Objective Approach to Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.06107 |