Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.06582 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916959462883328 |
|---|---|
| author | Wang, Steven H. Zubkov, Maksim Fan, Kexin Harrell, Sarah Sun, Yuyang Chen, Wei Plesner, Andreas Wattenhofer, Roger |
| author_facet | Wang, Steven H. Zubkov, Maksim Fan, Kexin Harrell, Sarah Sun, Yuyang Chen, Wei Plesner, Andreas Wattenhofer, Roger |
| contents | Information retrieval, specifically contract clause retrieval, is foundational to contract drafting because lawyers rarely draft contracts from scratch; instead, they locate and revise the most relevant precedent. We introduce the Atticus Clause Retrieval Dataset (ACORD), the first retrieval benchmark for contract drafting fully annotated by experts. ACORD focuses on complex contract clauses such as Limitation of Liability, Indemnification, Change of Control, and Most Favored Nation. It includes 114 queries and over 126,000 query-clause pairs, each ranked on a scale from 1 to 5 stars. The task is to find the most relevant precedent clauses to a query. The bi-encoder retriever paired with pointwise LLMs re-rankers shows promising results. However, substantial improvements are still needed to effectively manage the complex legal work typically undertaken by lawyers. As the first retrieval benchmark for contract drafting annotated by experts, ACORD can serve as a valuable IR benchmark for the NLP community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06582 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting Wang, Steven H. Zubkov, Maksim Fan, Kexin Harrell, Sarah Sun, Yuyang Chen, Wei Plesner, Andreas Wattenhofer, Roger Computation and Language Information retrieval, specifically contract clause retrieval, is foundational to contract drafting because lawyers rarely draft contracts from scratch; instead, they locate and revise the most relevant precedent. We introduce the Atticus Clause Retrieval Dataset (ACORD), the first retrieval benchmark for contract drafting fully annotated by experts. ACORD focuses on complex contract clauses such as Limitation of Liability, Indemnification, Change of Control, and Most Favored Nation. It includes 114 queries and over 126,000 query-clause pairs, each ranked on a scale from 1 to 5 stars. The task is to find the most relevant precedent clauses to a query. The bi-encoder retriever paired with pointwise LLMs re-rankers shows promising results. However, substantial improvements are still needed to effectively manage the complex legal work typically undertaken by lawyers. As the first retrieval benchmark for contract drafting annotated by experts, ACORD can serve as a valuable IR benchmark for the NLP community. |
| title | ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2501.06582 |