Jury: A Comprehensive Evaluation Toolkit
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913357071646720 |
|---|---|
| author | Cavusoglu, Devrim Sen, Secil Sert, Ulas Altinuc, Sinan |
| author_facet | Cavusoglu, Devrim Sen, Secil Sert, Ulas Altinuc, Sinan |
| contents | Evaluation plays a critical role in deep learning as a fundamental block of any prediction-based system. However, the vast number of Natural Language Processing (NLP) tasks and the development of various metrics have led to challenges in evaluating different systems with different metrics. To address these challenges, we introduce jury, a toolkit that provides a unified evaluation framework with standardized structures for performing evaluation across different tasks and metrics. The objective of jury is to standardize and improve metric evaluation for all systems and aid the community in overcoming the challenges in evaluation. Since its open-source release, jury has reached a wide audience and is available at https://github.com/obss/jury. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_02040 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Jury: A Comprehensive Evaluation Toolkit Cavusoglu, Devrim Sen, Secil Sert, Ulas Altinuc, Sinan Computation and Language Artificial Intelligence I.2.7; D.1.3 Evaluation plays a critical role in deep learning as a fundamental block of any prediction-based system. However, the vast number of Natural Language Processing (NLP) tasks and the development of various metrics have led to challenges in evaluating different systems with different metrics. To address these challenges, we introduce jury, a toolkit that provides a unified evaluation framework with standardized structures for performing evaluation across different tasks and metrics. The objective of jury is to standardize and improve metric evaluation for all systems and aid the community in overcoming the challenges in evaluation. Since its open-source release, jury has reached a wide audience and is available at https://github.com/obss/jury. |
| title | Jury: A Comprehensive Evaluation Toolkit |
| topic | Computation and Language Artificial Intelligence I.2.7; D.1.3 |
| url | https://arxiv.org/abs/2310.02040 |