Argument Quality Assessment in the Age of Instruction-Following Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917621208711168 |
|---|---|
| author | Wachsmuth, Henning Lapesa, Gabriella Cabrio, Elena Lauscher, Anne Park, Joonsuk Vecchi, Eva Maria Villata, Serena Ziegenbein, Timon |
| author_facet | Wachsmuth, Henning Lapesa, Gabriella Cabrio, Elena Lauscher, Anne Park, Joonsuk Vecchi, Eva Maria Villata, Serena Ziegenbein, Timon |
| contents | The computational treatment of arguments on controversial issues has been subject to extensive NLP research, due to its envisioned impact on opinion formation, decision making, writing education, and the like. A critical task in any such application is the assessment of an argument's quality - but it is also particularly challenging. In this position paper, we start from a brief survey of argument quality research, where we identify the diversity of quality notions and the subjectiveness of their perception as the main hurdles towards substantial progress on argument quality assessment. We argue that the capabilities of instruction-following large language models (LLMs) to leverage knowledge across contexts enable a much more reliable assessment. Rather than just fine-tuning LLMs towards leaderboard chasing on assessment tasks, they need to be instructed systematically with argumentation theories and scenarios as well as with ways to solve argument-related problems. We discuss the real-world opportunities and ethical issues emerging thereby. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16084 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Argument Quality Assessment in the Age of Instruction-Following Large Language Models Wachsmuth, Henning Lapesa, Gabriella Cabrio, Elena Lauscher, Anne Park, Joonsuk Vecchi, Eva Maria Villata, Serena Ziegenbein, Timon Computation and Language The computational treatment of arguments on controversial issues has been subject to extensive NLP research, due to its envisioned impact on opinion formation, decision making, writing education, and the like. A critical task in any such application is the assessment of an argument's quality - but it is also particularly challenging. In this position paper, we start from a brief survey of argument quality research, where we identify the diversity of quality notions and the subjectiveness of their perception as the main hurdles towards substantial progress on argument quality assessment. We argue that the capabilities of instruction-following large language models (LLMs) to leverage knowledge across contexts enable a much more reliable assessment. Rather than just fine-tuning LLMs towards leaderboard chasing on assessment tasks, they need to be instructed systematically with argumentation theories and scenarios as well as with ways to solve argument-related problems. We discuss the real-world opportunities and ethical issues emerging thereby. |
| title | Argument Quality Assessment in the Age of Instruction-Following Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.16084 |