LAMP: Extracting Local Decision Surfaces From Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914506929602560 |
|---|---|
| author | Chen, Ryan Ko, Youngmin Zhang, Zeyu Cho, Catherine Chung, Sunny Giuffré, Mauro Shung, Dennis L. Stadie, Bradly C. |
| author_facet | Chen, Ryan Ko, Youngmin Zhang, Zeyu Cho, Catherine Chung, Sunny Giuffré, Mauro Shung, Dennis L. Stadie, Bradly C. |
| contents | We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model's decision surface and studies how reliably a model maps its stated reasons to its reported predictions by approximating a decision surface. LAMP treats the model's own self-reported explanations as a coordinate system and fits a locally linear surrogate that links those weights to the model's output. By doing so, it reveals how much the stated factors steer the model's decisions. We apply LAMP to three tasks: sentiment analysis, controversial-topic detection, and safety-prompt auditing. Across these tasks, LAMP reveals that many language models' locally approximated linear decision landscapes overall agree with human judgments on explanation quality and, on a clinical case-file data set, align with expert assessments. Since LAMP operates without requiring access to model gradients, logits, or internal activations, it serves as a practical and lightweight framework for auditing proprietary language models, and enabling assessment of whether a model appears to behave consistently with the explanations it provides. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11772 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LAMP: Extracting Local Decision Surfaces From Large Language Models Chen, Ryan Ko, Youngmin Zhang, Zeyu Cho, Catherine Chung, Sunny Giuffré, Mauro Shung, Dennis L. Stadie, Bradly C. Machine Learning We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model's decision surface and studies how reliably a model maps its stated reasons to its reported predictions by approximating a decision surface. LAMP treats the model's own self-reported explanations as a coordinate system and fits a locally linear surrogate that links those weights to the model's output. By doing so, it reveals how much the stated factors steer the model's decisions. We apply LAMP to three tasks: sentiment analysis, controversial-topic detection, and safety-prompt auditing. Across these tasks, LAMP reveals that many language models' locally approximated linear decision landscapes overall agree with human judgments on explanation quality and, on a clinical case-file data set, align with expert assessments. Since LAMP operates without requiring access to model gradients, logits, or internal activations, it serves as a practical and lightweight framework for auditing proprietary language models, and enabling assessment of whether a model appears to behave consistently with the explanations it provides. |
| title | LAMP: Extracting Local Decision Surfaces From Large Language Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.11772 |