Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909167534473216 |
|---|---|
| author | Gallego, Victor |
| author_facet | Gallego, Victor |
| contents | This paper proposes an interpretation of RLAIF as Bayesian inference by introducing distilled Self-Critique (dSC), which refines the outputs of a LLM through a Gibbs sampler that is later distilled into a fine-tuned model. Only requiring synthetic data, dSC is exercised in experiments regarding safety, sentiment, and privacy control, showing it can be a viable and cheap alternative to align LLMs. Code released at \url{https://github.com/vicgalle/distilled-self-critique}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_01957 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective Gallego, Victor Computation and Language Machine Learning This paper proposes an interpretation of RLAIF as Bayesian inference by introducing distilled Self-Critique (dSC), which refines the outputs of a LLM through a Gibbs sampler that is later distilled into a fine-tuned model. Only requiring synthetic data, dSC is exercised in experiments regarding safety, sentiment, and privacy control, showing it can be a viable and cheap alternative to align LLMs. Code released at \url{https://github.com/vicgalle/distilled-self-critique}. |
| title | Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2312.01957 |