Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911842621718528 |
|---|---|
| author | Yuksekgonul, Mert Chandrasekaran, Varun Jones, Erik Gunasekar, Suriya Naik, Ranjita Palangi, Hamid Kamar, Ece Nushi, Besmira |
| author_facet | Yuksekgonul, Mert Chandrasekaran, Varun Jones, Erik Gunasekar, Suriya Naik, Ranjita Palangi, Hamid Kamar, Ece Nushi, Besmira |
| contents | We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the LLM interacts internally with factual constraints. We find a strong positive relationship between the LLM's attention to constraint tokens and the factual accuracy of generations. We curate a suite of 10 datasets containing over 40,000 prompts to study the task of predicting factual errors with the Llama-2 family across all scales (7B, 13B, 70B). We propose SAT Probe, a method probing attention patterns, that can predict factual errors and fine-grained constraint satisfaction, and allow early error identification. The approach and findings take another step towards using the mechanistic understanding of LLMs to enhance their reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_15098 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models Yuksekgonul, Mert Chandrasekaran, Varun Jones, Erik Gunasekar, Suriya Naik, Ranjita Palangi, Hamid Kamar, Ece Nushi, Besmira Computation and Language Artificial Intelligence Machine Learning We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the LLM interacts internally with factual constraints. We find a strong positive relationship between the LLM's attention to constraint tokens and the factual accuracy of generations. We curate a suite of 10 datasets containing over 40,000 prompts to study the task of predicting factual errors with the Llama-2 family across all scales (7B, 13B, 70B). We propose SAT Probe, a method probing attention patterns, that can predict factual errors and fine-grained constraint satisfaction, and allow early error identification. The approach and findings take another step towards using the mechanistic understanding of LLMs to enhance their reliability. |
| title | Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2309.15098 |