AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910006011494400 |
|---|---|
| author | Li, Yifei Zhang, Richong Tu, Wanyu Nie, Zhijie Luo, Haokun Yin, Chuantao Li, Pengchong |
| author_facet | Li, Yifei Zhang, Richong Tu, Wanyu Nie, Zhijie Luo, Haokun Yin, Chuantao Li, Pengchong |
| contents | Legal judgments may contain errors due to the complexity of case circumstances and the abstract nature of legal concepts, while existing appellate review mechanisms face efficiency pressures from a surge in case volumes. Although current legal AI research focuses on tasks like judgment prediction and legal document generation, the task of judgment review differs fundamentally in its objectives and paradigm: it centers on detecting, classifying, and correcting errors after a judgment is issued, constituting anomaly detection rather than prediction or generation. To address this research gap, we introduce a novel task APPELLATE REVIEW, aiming to assess models' diagnostic reasoning and reliability in legal practice. We also construct a novel dataset benchmark AR-BENCH, which comprises 8,700 finely annotated decisions and 34,617 supplementary corpora. By evaluating 14 large language models, we reveal critical limitations in existing models' ability to identify legal application errors, providing empirical evidence for future improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22742 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction Li, Yifei Zhang, Richong Tu, Wanyu Nie, Zhijie Luo, Haokun Yin, Chuantao Li, Pengchong Computation and Language Legal judgments may contain errors due to the complexity of case circumstances and the abstract nature of legal concepts, while existing appellate review mechanisms face efficiency pressures from a surge in case volumes. Although current legal AI research focuses on tasks like judgment prediction and legal document generation, the task of judgment review differs fundamentally in its objectives and paradigm: it centers on detecting, classifying, and correcting errors after a judgment is issued, constituting anomaly detection rather than prediction or generation. To address this research gap, we introduce a novel task APPELLATE REVIEW, aiming to assess models' diagnostic reasoning and reliability in legal practice. We also construct a novel dataset benchmark AR-BENCH, which comprises 8,700 finely annotated decisions and 34,617 supplementary corpora. By evaluating 14 large language models, we reveal critical limitations in existing models' ability to identify legal application errors, providing empirical evidence for future improvements. |
| title | AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.22742 |