Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2406.17186 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909232241049600 |
|---|---|
| author | Hou, Abe Bohan Weller, Orion Qin, Guanghui Yang, Eugene Lawrie, Dawn Holzenberger, Nils Blair-Stanek, Andrew Van Durme, Benjamin |
| author_facet | Hou, Abe Bohan Weller, Orion Qin, Guanghui Yang, Eugene Lawrie, Dawn Holzenberger, Nils Blair-Stanek, Andrew Van Durme, Benjamin |
| contents | Legal professionals need to write analyses that rely on citations to relevant precedents, i.e., previous case decisions. Intelligent systems assisting legal professionals in writing such documents provide great benefits but are challenging to design. Such systems need to help locate, summarize, and reason over salient precedents in order to be useful. To enable systems for such tasks, we work with legal professionals to transform a large open-source legal corpus into a dataset supporting two important backbone tasks: information retrieval (IR) and retrieval-augmented generation (RAG). This dataset CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal. We benchmark state-of-the-art models on CLERC, showing that current approaches still struggle: GPT-4o generates analyses with the highest ROUGE F-scores but hallucinates the most, while zero-shot IR models only achieve 48.3% recall@1000. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_17186 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation Hou, Abe Bohan Weller, Orion Qin, Guanghui Yang, Eugene Lawrie, Dawn Holzenberger, Nils Blair-Stanek, Andrew Van Durme, Benjamin Computation and Language Computers and Society Legal professionals need to write analyses that rely on citations to relevant precedents, i.e., previous case decisions. Intelligent systems assisting legal professionals in writing such documents provide great benefits but are challenging to design. Such systems need to help locate, summarize, and reason over salient precedents in order to be useful. To enable systems for such tasks, we work with legal professionals to transform a large open-source legal corpus into a dataset supporting two important backbone tasks: information retrieval (IR) and retrieval-augmented generation (RAG). This dataset CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal. We benchmark state-of-the-art models on CLERC, showing that current approaches still struggle: GPT-4o generates analyses with the highest ROUGE F-scores but hallucinates the most, while zero-shot IR models only achieve 48.3% recall@1000. |
| title | CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2406.17186 |