Autoformalizing Natural Language to First-Order Logic: A Case Study in Logical Fallacy Detection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913720823709696 |
|---|---|
| author | Lalwani, Abhinav Kim, Tasha Chopra, Lovish Hahn, Christopher Jin, Zhijing Sachan, Mrinmaya |
| author_facet | Lalwani, Abhinav Kim, Tasha Chopra, Lovish Hahn, Christopher Jin, Zhijing Sachan, Mrinmaya |
| contents | Translating natural language into formal language such as First-Order Logic (FOL) is a foundational challenge in NLP with wide-ranging applications in automated reasoning, misinformation tracking, and knowledge validation. In this paper, we introduce Natural Language to First-Order Logic (NL2FOL), a framework to autoformalize natural language to FOL step by step using Large Language Models (LLMs). Our approach addresses key challenges in this translation process, including the integration of implicit background knowledge. By leveraging structured representations generated by NL2FOL, we use Satisfiability Modulo Theory (SMT) solvers to reason about the logical validity of natural language statements. We present logical fallacy detection as a case study to evaluate the efficacy of NL2FOL. Being neurosymbolic, our approach also provides interpretable insights into the reasoning process and demonstrates robustness without requiring model fine-tuning or labeled training data. Our framework achieves strong performance on multiple datasets. On the LOGIC dataset, NL2FOL achieves an F1-score of 78%, while generalizing effectively to the LOGICCLIMATE dataset with an F1-score of 80%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_02318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Autoformalizing Natural Language to First-Order Logic: A Case Study in Logical Fallacy Detection Lalwani, Abhinav Kim, Tasha Chopra, Lovish Hahn, Christopher Jin, Zhijing Sachan, Mrinmaya Computation and Language Artificial Intelligence Machine Learning Logic in Computer Science Translating natural language into formal language such as First-Order Logic (FOL) is a foundational challenge in NLP with wide-ranging applications in automated reasoning, misinformation tracking, and knowledge validation. In this paper, we introduce Natural Language to First-Order Logic (NL2FOL), a framework to autoformalize natural language to FOL step by step using Large Language Models (LLMs). Our approach addresses key challenges in this translation process, including the integration of implicit background knowledge. By leveraging structured representations generated by NL2FOL, we use Satisfiability Modulo Theory (SMT) solvers to reason about the logical validity of natural language statements. We present logical fallacy detection as a case study to evaluate the efficacy of NL2FOL. Being neurosymbolic, our approach also provides interpretable insights into the reasoning process and demonstrates robustness without requiring model fine-tuning or labeled training data. Our framework achieves strong performance on multiple datasets. On the LOGIC dataset, NL2FOL achieves an F1-score of 78%, while generalizing effectively to the LOGICCLIMATE dataset with an F1-score of 80%. |
| title | Autoformalizing Natural Language to First-Order Logic: A Case Study in Logical Fallacy Detection |
| topic | Computation and Language Artificial Intelligence Machine Learning Logic in Computer Science |
| url | https://arxiv.org/abs/2405.02318 |