Legal Evalutions and Challenges of Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913578789896192 |
|---|---|
| author | Wang, Jiaqi Zhao, Huan Yang, Zhenyuan Shu, Peng Chen, Junhao Sun, Haobo Liang, Ruixi Li, Shixin Shi, Pengcheng Ma, Longjun Liu, Zongjia Liu, Zhengliang Zhong, Tianyang Zhang, Yutong Ma, Chong Zhang, Xin Zhang, Tuo Ding, Tianli Ren, Yudan Liu, Tianming Jiang, Xi Zhang, Shu |
| author_facet | Wang, Jiaqi Zhao, Huan Yang, Zhenyuan Shu, Peng Chen, Junhao Sun, Haobo Liang, Ruixi Li, Shixin Shi, Pengcheng Ma, Longjun Liu, Zongjia Liu, Zhengliang Zhong, Tianyang Zhang, Yutong Ma, Chong Zhang, Xin Zhang, Tuo Ding, Tianli Ren, Yudan Liu, Tianming Jiang, Xi Zhang, Shu |
| contents | In this paper, we review legal testing methods based on Large Language Models (LLMs), using the OPENAI o1 model as a case study to evaluate the performance of large models in applying legal provisions. We compare current state-of-the-art LLMs, including open-source, closed-source, and legal-specific models trained specifically for the legal domain. Systematic tests are conducted on English and Chinese legal cases, and the results are analyzed in depth. Through systematic testing of legal cases from common law systems and China, this paper explores the strengths and weaknesses of LLMs in understanding and applying legal texts, reasoning through legal issues, and predicting judgments. The experimental results highlight both the potential and limitations of LLMs in legal applications, particularly in terms of challenges related to the interpretation of legal language and the accuracy of legal reasoning. Finally, the paper provides a comprehensive analysis of the advantages and disadvantages of various types of models, offering valuable insights and references for the future application of AI in the legal field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10137 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Legal Evalutions and Challenges of Large Language Models Wang, Jiaqi Zhao, Huan Yang, Zhenyuan Shu, Peng Chen, Junhao Sun, Haobo Liang, Ruixi Li, Shixin Shi, Pengcheng Ma, Longjun Liu, Zongjia Liu, Zhengliang Zhong, Tianyang Zhang, Yutong Ma, Chong Zhang, Xin Zhang, Tuo Ding, Tianli Ren, Yudan Liu, Tianming Jiang, Xi Zhang, Shu Computation and Language Artificial Intelligence In this paper, we review legal testing methods based on Large Language Models (LLMs), using the OPENAI o1 model as a case study to evaluate the performance of large models in applying legal provisions. We compare current state-of-the-art LLMs, including open-source, closed-source, and legal-specific models trained specifically for the legal domain. Systematic tests are conducted on English and Chinese legal cases, and the results are analyzed in depth. Through systematic testing of legal cases from common law systems and China, this paper explores the strengths and weaknesses of LLMs in understanding and applying legal texts, reasoning through legal issues, and predicting judgments. The experimental results highlight both the potential and limitations of LLMs in legal applications, particularly in terms of challenges related to the interpretation of legal language and the accuracy of legal reasoning. Finally, the paper provides a comprehensive analysis of the advantages and disadvantages of various types of models, offering valuable insights and references for the future application of AI in the legal field. |
| title | Legal Evalutions and Challenges of Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.10137 |