Representation Surgery: Theory and Practice of Affine Steering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915322589609984 |
|---|---|
| author | Singh, Shashwat Ravfogel, Shauli Herzig, Jonathan Aharoni, Roee Cotterell, Ryan Kumaraguru, Ponnurangam |
| author_facet | Singh, Shashwat Ravfogel, Shauli Herzig, Jonathan Aharoni, Roee Cotterell, Ryan Kumaraguru, Ponnurangam |
| contents | Language models often exhibit undesirable behavior, e.g., generating toxic or gender-biased text. In the case of neural language models, an encoding of the undesirable behavior is often present in the model's representations. Thus, one natural (and common) approach to prevent the model from exhibiting undesirable behavior is to steer the model's representations in a manner that reduces the probability of it generating undesirable text. This paper investigates the formal and empirical properties of steering functions, i.e., transformation of the neural language model's representations that alter its behavior. First, we derive two optimal, in the least-squares sense, affine steering functions under different constraints. Our theory provides justification for existing approaches and offers a novel, improved steering approach. Second, we offer a series of experiments that demonstrate the empirical effectiveness of the methods in mitigating bias and reducing toxic generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_09631 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Representation Surgery: Theory and Practice of Affine Steering Singh, Shashwat Ravfogel, Shauli Herzig, Jonathan Aharoni, Roee Cotterell, Ryan Kumaraguru, Ponnurangam Machine Learning Computation and Language Computers and Society Language models often exhibit undesirable behavior, e.g., generating toxic or gender-biased text. In the case of neural language models, an encoding of the undesirable behavior is often present in the model's representations. Thus, one natural (and common) approach to prevent the model from exhibiting undesirable behavior is to steer the model's representations in a manner that reduces the probability of it generating undesirable text. This paper investigates the formal and empirical properties of steering functions, i.e., transformation of the neural language model's representations that alter its behavior. First, we derive two optimal, in the least-squares sense, affine steering functions under different constraints. Our theory provides justification for existing approaches and offers a novel, improved steering approach. Second, we offer a series of experiments that demonstrate the empirical effectiveness of the methods in mitigating bias and reducing toxic generation. |
| title | Representation Surgery: Theory and Practice of Affine Steering |
| topic | Machine Learning Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2402.09631 |