From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908382030462976 |
|---|---|
| author | Yu, Stanley Bulusu, Vaidehi Yasunaga, Oscar Lau, Clayton Blondin, Cole O'Brien, Sean Zhu, Kevin Sharma, Vasu |
| author_facet | Yu, Stanley Bulusu, Vaidehi Yasunaga, Oscar Lau, Clayton Blondin, Cole O'Brien, Sean Zhu, Kevin Sharma, Vasu |
| contents | Large Language Models (LLMs) exhibit strong conversational abilities but often generate falsehoods. Prior work suggests that the truthfulness of simple propositions can be represented as a single linear direction in a model's internal activations, but this may not fully capture its underlying geometry. In this work, we extend the concept cone framework, recently introduced for modeling refusal, to the domain of truth. We identify multi-dimensional cones that causally mediate truth-related behavior across multiple LLM families. Our results are supported by three lines of evidence: (i) causal interventions reliably flip model responses to factual statements, (ii) learned cones generalize across model architectures, and (iii) cone-based interventions preserve unrelated model behavior. These findings reveal the richer, multidirectional structure governing simple true/false propositions in LLMs and highlight concept cones as a promising tool for probing abstract behaviors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs Yu, Stanley Bulusu, Vaidehi Yasunaga, Oscar Lau, Clayton Blondin, Cole O'Brien, Sean Zhu, Kevin Sharma, Vasu Machine Learning Computation and Language Large Language Models (LLMs) exhibit strong conversational abilities but often generate falsehoods. Prior work suggests that the truthfulness of simple propositions can be represented as a single linear direction in a model's internal activations, but this may not fully capture its underlying geometry. In this work, we extend the concept cone framework, recently introduced for modeling refusal, to the domain of truth. We identify multi-dimensional cones that causally mediate truth-related behavior across multiple LLM families. Our results are supported by three lines of evidence: (i) causal interventions reliably flip model responses to factual statements, (ii) learned cones generalize across model architectures, and (iii) cone-based interventions preserve unrelated model behavior. These findings reveal the richer, multidirectional structure governing simple true/false propositions in LLMs and highlight concept cones as a promising tool for probing abstract behaviors. |
| title | From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2505.21800 |