Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.08098 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929560981864448 |
|---|---|
| author | Xie, Huiyuan Steffek, Felix de Faria, Joana Ribeiro Carter, Christine Rutherford, Jonathan |
| author_facet | Xie, Huiyuan Steffek, Felix de Faria, Joana Ribeiro Carter, Christine Rutherford, Jonathan |
| contents | This paper explores the intersection of technological innovation and access to justice by developing a benchmark for predicting case outcomes in the UK Employment Tribunal (UKET). To address the challenge of extensive manual annotation, the study employs a large language model (LLM) for automatic annotation, resulting in the creation of the CLC-UKET dataset. The dataset consists of approximately 19,000 UKET cases and their metadata. Comprehensive legal annotations cover facts, claims, precedent references, statutory references, case outcomes, reasons and jurisdiction codes. Facilitated by the CLC-UKET data, we examine a multi-class case outcome prediction task in the UKET. Human predictions are collected to establish a performance reference for model comparison. Empirical results from baseline models indicate that finetuned transformer models outperform zero-shot and few-shot LLMs on the UKET prediction task. The performance of zero-shot LLMs can be enhanced by integrating task-related information into few-shot examples. We hope that the CLC-UKET dataset, along with human annotations and empirical findings, can serve as a valuable benchmark for employment-related dispute resolution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_08098 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal Xie, Huiyuan Steffek, Felix de Faria, Joana Ribeiro Carter, Christine Rutherford, Jonathan Computation and Language Artificial Intelligence This paper explores the intersection of technological innovation and access to justice by developing a benchmark for predicting case outcomes in the UK Employment Tribunal (UKET). To address the challenge of extensive manual annotation, the study employs a large language model (LLM) for automatic annotation, resulting in the creation of the CLC-UKET dataset. The dataset consists of approximately 19,000 UKET cases and their metadata. Comprehensive legal annotations cover facts, claims, precedent references, statutory references, case outcomes, reasons and jurisdiction codes. Facilitated by the CLC-UKET data, we examine a multi-class case outcome prediction task in the UKET. Human predictions are collected to establish a performance reference for model comparison. Empirical results from baseline models indicate that finetuned transformer models outperform zero-shot and few-shot LLMs on the UKET prediction task. The performance of zero-shot LLMs can be enhanced by integrating task-related information into few-shot examples. We hope that the CLC-UKET dataset, along with human annotations and empirical findings, can serve as a valuable benchmark for employment-related dispute resolution. |
| title | The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.08098 |