Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.11396 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909350003474432 |
|---|---|
| author | Thuy, Phan Thi Thanh Yamamoto, Akihiro |
| author_facet | Thuy, Phan Thi Thanh Yamamoto, Akihiro |
| contents | In this paper we propose that a restricted version of logical inference can be implemented with self-attention networks. We are aiming at showing that LLMs (Large Language Models) constructed with transformer networks can make logical inferences. We would reveal the potential of LLMs by analyzing self-attention networks, which are main components of transformer networks. Our approach is not based on semantics of natural languages but operations of logical inference. %point of view. We show that hierarchical constructions of self-attention networks with feed forward networks (FFNs) can implement top-down derivations for a class of logical formulae. We also show bottom-up derivations are also implemented for the same class. We believe that our results show that LLMs implicitly have the power of logical inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_11396 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Implementing Derivations of Definite Logic Programs with Self-Attention Networks Thuy, Phan Thi Thanh Yamamoto, Akihiro Artificial Intelligence In this paper we propose that a restricted version of logical inference can be implemented with self-attention networks. We are aiming at showing that LLMs (Large Language Models) constructed with transformer networks can make logical inferences. We would reveal the potential of LLMs by analyzing self-attention networks, which are main components of transformer networks. Our approach is not based on semantics of natural languages but operations of logical inference. %point of view. We show that hierarchical constructions of self-attention networks with feed forward networks (FFNs) can implement top-down derivations for a class of logical formulae. We also show bottom-up derivations are also implemented for the same class. We believe that our results show that LLMs implicitly have the power of logical inference. |
| title | Implementing Derivations of Definite Logic Programs with Self-Attention Networks |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2410.11396 |