Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908623476621312 |
|---|---|
| author | Boggust, Angie Bang, Hyemin Strobelt, Hendrik Satyanarayan, Arvind |
| author_facet | Boggust, Angie Bang, Hyemin Strobelt, Hendrik Satyanarayan, Arvind |
| contents | While interpretability methods identify a model's learned concepts, they overlook the relationships between concepts that make up its abstractions and inform its ability to generalize to new data. To assess whether models' have learned human-aligned abstractions, we introduce abstraction alignment, a methodology to compare model behavior against formal human knowledge. Abstraction alignment externalizes domain-specific human knowledge as an abstraction graph, a set of pertinent concepts spanning levels of abstraction. Using the abstraction graph as a ground truth, abstraction alignment measures the alignment of a model's behavior by determining how much of its uncertainty is accounted for by the human abstractions. By aggregating abstraction alignment across entire datasets, users can test alignment hypotheses, such as which human concepts the model has learned and where misalignments recur. In evaluations with experts, abstraction alignment differentiates seemingly similar errors, improves the verbosity of existing model-quality metrics, and uncovers improvements to current human abstractions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12543 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships Boggust, Angie Bang, Hyemin Strobelt, Hendrik Satyanarayan, Arvind Machine Learning Artificial Intelligence Computation and Language Human-Computer Interaction While interpretability methods identify a model's learned concepts, they overlook the relationships between concepts that make up its abstractions and inform its ability to generalize to new data. To assess whether models' have learned human-aligned abstractions, we introduce abstraction alignment, a methodology to compare model behavior against formal human knowledge. Abstraction alignment externalizes domain-specific human knowledge as an abstraction graph, a set of pertinent concepts spanning levels of abstraction. Using the abstraction graph as a ground truth, abstraction alignment measures the alignment of a model's behavior by determining how much of its uncertainty is accounted for by the human abstractions. By aggregating abstraction alignment across entire datasets, users can test alignment hypotheses, such as which human concepts the model has learned and where misalignments recur. In evaluations with experts, abstraction alignment differentiates seemingly similar errors, improves the verbosity of existing model-quality metrics, and uncovers improvements to current human abstractions. |
| title | Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships |
| topic | Machine Learning Artificial Intelligence Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2407.12543 |