Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.00847 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914082013052928 |
|---|---|
| author | Altemeyer, Moritz Eger, Steffen Daxenberger, Johannes Chen, Yanran Altendorf, Tim Cimiano, Philipp Schiller, Benjamin |
| author_facet | Altemeyer, Moritz Eger, Steffen Daxenberger, Johannes Chen, Yanran Altendorf, Tim Cimiano, Philipp Schiller, Benjamin |
| contents | Large Language Models (LLMs) have revolutionized various Natural Language Generation (NLG) tasks, including Argument Summarization (ArgSum), a key subfield of Argument Mining. This paper investigates the integration of state-of-the-art LLMs into ArgSum systems and their evaluation. In particular, we propose a novel prompt-based evaluation scheme, and validate it through a novel human benchmark dataset. Our work makes three main contributions: (i) the integration of LLMs into existing ArgSum systems, (ii) the development of two new LLM-based ArgSum systems, benchmarked against prior methods, and (iii) the introduction of an advanced LLM-based evaluation scheme. We demonstrate that the use of LLMs substantially improves both the generation and evaluation of argument summaries, achieving state-of-the-art results and advancing the field of ArgSum. We also show that among the four LLMs integrated in (i) and (ii), Qwen-3-32B, despite having the fewest parameters, performs best, even surpassing GPT-4o. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00847 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Argument Summarization and its Evaluation in the Era of Large Language Models Altemeyer, Moritz Eger, Steffen Daxenberger, Johannes Chen, Yanran Altendorf, Tim Cimiano, Philipp Schiller, Benjamin Computation and Language Large Language Models (LLMs) have revolutionized various Natural Language Generation (NLG) tasks, including Argument Summarization (ArgSum), a key subfield of Argument Mining. This paper investigates the integration of state-of-the-art LLMs into ArgSum systems and their evaluation. In particular, we propose a novel prompt-based evaluation scheme, and validate it through a novel human benchmark dataset. Our work makes three main contributions: (i) the integration of LLMs into existing ArgSum systems, (ii) the development of two new LLM-based ArgSum systems, benchmarked against prior methods, and (iii) the introduction of an advanced LLM-based evaluation scheme. We demonstrate that the use of LLMs substantially improves both the generation and evaluation of argument summaries, achieving state-of-the-art results and advancing the field of ArgSum. We also show that among the four LLMs integrated in (i) and (ii), Qwen-3-32B, despite having the fewest parameters, performs best, even surpassing GPT-4o. |
| title | Argument Summarization and its Evaluation in the Era of Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.00847 |