Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.02976 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909063310213120 |
|---|---|
| author | Li, Jiahang Chen, Taoyu Wang, Yuanli |
| author_facet | Li, Jiahang Chen, Taoyu Wang, Yuanli |
| contents | This study introduces a novel approach for analyzing and modifying entity relationships in GPT models, diverging from ROME's entity-focused methods. We develop a relation tracing technique to understand the influence of language model computations on relationship judgments. Using the FewRel dataset, we identify key roles of MLP modules and attention mechanisms in processing relationship information. Our method, tested against ROME on a new dataset, shows improved balance in specificity and generalization, underscoring the potential of manipulating early-layer modules for enhanced model understanding and accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02976 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Trace and Edit Relation Associations in GPT Li, Jiahang Chen, Taoyu Wang, Yuanli Computation and Language Artificial Intelligence This study introduces a novel approach for analyzing and modifying entity relationships in GPT models, diverging from ROME's entity-focused methods. We develop a relation tracing technique to understand the influence of language model computations on relationship judgments. Using the FewRel dataset, we identify key roles of MLP modules and attention mechanisms in processing relationship information. Our method, tested against ROME on a new dataset, shows improved balance in specificity and generalization, underscoring the potential of manipulating early-layer modules for enhanced model understanding and accuracy. |
| title | Trace and Edit Relation Associations in GPT |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2401.02976 |