Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.07849 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910246997327872 |
|---|---|
| author | Singh, Amrita Karaca, H. Suhan Joshi, Aditya Paik, Hye-young Jiang, Jiaojiao |
| author_facet | Singh, Amrita Karaca, H. Suhan Joshi, Aditya Paik, Hye-young Jiang, Jiaojiao |
| contents | Despite advances in legal NLP, no comprehensive evaluation of Transformer-based models customized for legal tasks (referred to as `legal-specific' models in this paper) exists for contract classification tasks. To address this gap, we present an evaluation of 13 legal-specific transformer-based models on 3 English-language contract classification tasks and compare them with 9 generalist models. The results show that legal-specific models consistently outperform generalist models, especially on tasks requiring nuanced legal understanding. They also help reduce misclassification of rare classes in imbalanced datasets. Legal-BERT and Contracts-BERT establish new SOTAs on two of the three tasks, despite having 69% fewer parameters than the best-performing generalist models. We also identify CaseLaw-BERT and LexLM as strong additional baselines for contract classification. Our results highlight the shortcomings of generalist models, emphasizing the need for domain-specific customization, particularly in the context of legal applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_07849 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classification Singh, Amrita Karaca, H. Suhan Joshi, Aditya Paik, Hye-young Jiang, Jiaojiao Computation and Language Despite advances in legal NLP, no comprehensive evaluation of Transformer-based models customized for legal tasks (referred to as `legal-specific' models in this paper) exists for contract classification tasks. To address this gap, we present an evaluation of 13 legal-specific transformer-based models on 3 English-language contract classification tasks and compare them with 9 generalist models. The results show that legal-specific models consistently outperform generalist models, especially on tasks requiring nuanced legal understanding. They also help reduce misclassification of rare classes in imbalanced datasets. Legal-BERT and Contracts-BERT establish new SOTAs on two of the three tasks, despite having 69% fewer parameters than the best-performing generalist models. We also identify CaseLaw-BERT and LexLM as strong additional baselines for contract classification. Our results highlight the shortcomings of generalist models, emphasizing the need for domain-specific customization, particularly in the context of legal applications. |
| title | Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classification |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.07849 |