Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910492938731520 |
|---|---|
| author | Robbani, Irfan Reisert, Paul Inoue, Naoya Pothong, Surawat Guerraoui, Camélia Wang, Wenzhi Naito, Shoichi Choi, Jungmin Inui, Kentaro |
| author_facet | Robbani, Irfan Reisert, Paul Inoue, Naoya Pothong, Surawat Guerraoui, Camélia Wang, Wenzhi Naito, Shoichi Choi, Jungmin Inui, Kentaro |
| contents | Prior research in computational argumentation has mainly focused on scoring the quality of arguments, with less attention on explicating logical errors. In this work, we introduce four sets of explainable templates for common informal logical fallacies designed to explicate a fallacy's implicit logic. Using our templates, we conduct an annotation study on top of 400 fallacious arguments taken from LOGIC dataset and achieve a high agreement score (Krippendorf's alpha of 0.54) and reasonable coverage (0.83). Finally, we conduct an experiment for detecting the structure of fallacies and discover that state-of-the-art language models struggle with detecting fallacy templates (0.47 accuracy). To facilitate research on fallacies, we make our dataset and guidelines publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12402 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling Robbani, Irfan Reisert, Paul Inoue, Naoya Pothong, Surawat Guerraoui, Camélia Wang, Wenzhi Naito, Shoichi Choi, Jungmin Inui, Kentaro Computation and Language Prior research in computational argumentation has mainly focused on scoring the quality of arguments, with less attention on explicating logical errors. In this work, we introduce four sets of explainable templates for common informal logical fallacies designed to explicate a fallacy's implicit logic. Using our templates, we conduct an annotation study on top of 400 fallacious arguments taken from LOGIC dataset and achieve a high agreement score (Krippendorf's alpha of 0.54) and reasonable coverage (0.83). Finally, we conduct an experiment for detecting the structure of fallacies and discover that state-of-the-art language models struggle with detecting fallacy templates (0.47 accuracy). To facilitate research on fallacies, we make our dataset and guidelines publicly available. |
| title | Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.12402 |