Supporting Humans in Evaluating AI Summaries of Legal Depositions
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915744507232256 |
|---|---|
| author | Farzi, Naghmeh Dietz, Laura Lewis, Dave D. |
| author_facet | Farzi, Naghmeh Dietz, Laura Lewis, Dave D. |
| contents | While large language models (LLMs) are increasingly used to summarize long documents, this trend poses significant challenges in the legal domain, where the factual accuracy of deposition summaries is crucial. Nugget-based methods have been shown to be extremely helpful for the automated evaluation of summarization approaches. In this work, we translate these methods to the user side and explore how nuggets could directly assist end users. Although prior systems have demonstrated the promise of nugget-based evaluation, its potential to support end users remains underexplored. Focusing on the legal domain, we present a prototype that leverages a factual nugget-based approach to support legal professionals in two concrete scenarios: (1) determining which of two summaries is better, and (2) manually improving an automatically generated summary. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_15182 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Supporting Humans in Evaluating AI Summaries of Legal Depositions Farzi, Naghmeh Dietz, Laura Lewis, Dave D. Computation and Language Information Retrieval H.3 While large language models (LLMs) are increasingly used to summarize long documents, this trend poses significant challenges in the legal domain, where the factual accuracy of deposition summaries is crucial. Nugget-based methods have been shown to be extremely helpful for the automated evaluation of summarization approaches. In this work, we translate these methods to the user side and explore how nuggets could directly assist end users. Although prior systems have demonstrated the promise of nugget-based evaluation, its potential to support end users remains underexplored. Focusing on the legal domain, we present a prototype that leverages a factual nugget-based approach to support legal professionals in two concrete scenarios: (1) determining which of two summaries is better, and (2) manually improving an automatically generated summary. |
| title | Supporting Humans in Evaluating AI Summaries of Legal Depositions |
| topic | Computation and Language Information Retrieval H.3 |
| url | https://arxiv.org/abs/2601.15182 |