Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.10275 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929278191403008 |
|---|---|
| author | Bogaert, Jeremie Standaert, Francois-Xavier |
| author_facet | Bogaert, Jeremie Standaert, Francois-Xavier |
| contents | The explanations of large language models have recently been shown to be sensitive to the randomness used for their training, creating a need to characterize this sensitivity. In this paper, we propose a characterization that questions the possibility to provide simple and informative explanations for such models. To this end, we give statistical definitions for the explanations' signal, noise and signal-to-noise ratio. We highlight that, in a typical case study where word-level univariate explanations are analyzed with first-order statistical tools, the explanations of simple feature-based models carry more signal and less noise than those of transformer ones. We then discuss the possibility to improve these results with alternative definitions of signal and noise that would capture more complex explanations and analysis methods, while also questioning the tradeoff with their plausibility for readers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10275 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Question on the Explainability of Large Language Models and the Word-Level Univariate First-Order Plausibility Assumption Bogaert, Jeremie Standaert, Francois-Xavier Computation and Language Artificial Intelligence The explanations of large language models have recently been shown to be sensitive to the randomness used for their training, creating a need to characterize this sensitivity. In this paper, we propose a characterization that questions the possibility to provide simple and informative explanations for such models. To this end, we give statistical definitions for the explanations' signal, noise and signal-to-noise ratio. We highlight that, in a typical case study where word-level univariate explanations are analyzed with first-order statistical tools, the explanations of simple feature-based models carry more signal and less noise than those of transformer ones. We then discuss the possibility to improve these results with alternative definitions of signal and noise that would capture more complex explanations and analysis methods, while also questioning the tradeoff with their plausibility for readers. |
| title | A Question on the Explainability of Large Language Models and the Word-Level Univariate First-Order Plausibility Assumption |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.10275 |