The Right Model for the Job: An Evaluation of Legal Multi-Label Classification Baselines
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916101438308352 |
|---|---|
| author | Forster, Martina Schulz, Claudia Nokku, Prudhvi Mirsafian, Melicaalsadat Kasundra, Jaykumar Skylaki, Stavroula |
| author_facet | Forster, Martina Schulz, Claudia Nokku, Prudhvi Mirsafian, Melicaalsadat Kasundra, Jaykumar Skylaki, Stavroula |
| contents | Multi-Label Classification (MLC) is a common task in the legal domain, where more than one label may be assigned to a legal document. A wide range of methods can be applied, ranging from traditional ML approaches to the latest Transformer-based architectures. In this work, we perform an evaluation of different MLC methods using two public legal datasets, POSTURE50K and EURLEX57K. By varying the amount of training data and the number of labels, we explore the comparative advantage offered by different approaches in relation to the dataset properties. Our findings highlight DistilRoBERTa and LegalBERT as performing consistently well in legal MLC with reasonable computational demands. T5 also demonstrates comparable performance while offering advantages as a generative model in the presence of changing label sets. Finally, we show that the CrossEncoder exhibits potential for notable macro-F1 score improvements, albeit with increased computational costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_11852 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Right Model for the Job: An Evaluation of Legal Multi-Label Classification Baselines Forster, Martina Schulz, Claudia Nokku, Prudhvi Mirsafian, Melicaalsadat Kasundra, Jaykumar Skylaki, Stavroula Computation and Language Artificial Intelligence Multi-Label Classification (MLC) is a common task in the legal domain, where more than one label may be assigned to a legal document. A wide range of methods can be applied, ranging from traditional ML approaches to the latest Transformer-based architectures. In this work, we perform an evaluation of different MLC methods using two public legal datasets, POSTURE50K and EURLEX57K. By varying the amount of training data and the number of labels, we explore the comparative advantage offered by different approaches in relation to the dataset properties. Our findings highlight DistilRoBERTa and LegalBERT as performing consistently well in legal MLC with reasonable computational demands. T5 also demonstrates comparable performance while offering advantages as a generative model in the presence of changing label sets. Finally, we show that the CrossEncoder exhibits potential for notable macro-F1 score improvements, albeit with increased computational costs. |
| title | The Right Model for the Job: An Evaluation of Legal Multi-Label Classification Baselines |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2401.11852 |