STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910497777909760 |
|---|---|
| author | Su, Weihang Hu, Yiran Xie, Anzhe Ai, Qingyao Que, Zibing Zheng, Ning Liu, Yun Shen, Weixing Liu, Yiqun |
| author_facet | Su, Weihang Hu, Yiran Xie, Anzhe Ai, Qingyao Que, Zibing Zheng, Ning Liu, Yun Shen, Weixing Liu, Yiqun |
| contents | Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated judicial decisions, legal document drafting, etc. Existing statute retrieval benchmarks focus on formal and professional queries from sources like bar exams and legal case documents, thereby neglecting non-professional queries from the general public, which often lack precise legal terminology and references. To address this gap, we introduce the STAtute Retrieval Dataset (STARD), a Chinese dataset comprising 1,543 query cases collected from real-world legal consultations and 55,348 candidate statutory articles. Unlike existing statute retrieval datasets, which primarily focus on professional legal queries, STARD captures the complexity and diversity of real queries from the general public. Through a comprehensive evaluation of various retrieval baselines, we reveal that existing retrieval approaches all fall short of these real queries issued by non-professional users. The best method only achieves a Recall@100 of 0.907, suggesting the necessity for further exploration and additional research in this area.
All the codes and datasets are available at: https://github.com/oneal2000/STARD/tree/main |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15313 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals Su, Weihang Hu, Yiran Xie, Anzhe Ai, Qingyao Que, Zibing Zheng, Ning Liu, Yun Shen, Weixing Liu, Yiqun Information Retrieval Computation and Language Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated judicial decisions, legal document drafting, etc. Existing statute retrieval benchmarks focus on formal and professional queries from sources like bar exams and legal case documents, thereby neglecting non-professional queries from the general public, which often lack precise legal terminology and references. To address this gap, we introduce the STAtute Retrieval Dataset (STARD), a Chinese dataset comprising 1,543 query cases collected from real-world legal consultations and 55,348 candidate statutory articles. Unlike existing statute retrieval datasets, which primarily focus on professional legal queries, STARD captures the complexity and diversity of real queries from the general public. Through a comprehensive evaluation of various retrieval baselines, we reveal that existing retrieval approaches all fall short of these real queries issued by non-professional users. The best method only achieves a Recall@100 of 0.907, suggesting the necessity for further exploration and additional research in this area. All the codes and datasets are available at: https://github.com/oneal2000/STARD/tree/main |
| title | STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2406.15313 |