EvalCards: A Framework for Standardized Evaluation Reporting
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917107410665472 |
|---|---|
| author | Dhar, Ruchira Villegas, Danae Sanchez Karamolegkou, Antonia Schiavone, Alice Yuan, Yifei Chen, Xinyi Li, Jiaang Frank, Stella De Grazia, Laura Swain, Monorama Brandl, Stephanie Hershcovich, Daniel Søgaard, Anders Elliott, Desmond |
| author_facet | Dhar, Ruchira Villegas, Danae Sanchez Karamolegkou, Antonia Schiavone, Alice Yuan, Yifei Chen, Xinyi Li, Jiaang Frank, Stella De Grazia, Laura Swain, Monorama Brandl, Stephanie Hershcovich, Daniel Søgaard, Anders Elliott, Desmond |
| contents | Evaluation has long been a central concern in NLP, and transparent reporting practices are more critical than ever in today's landscape of rapidly released open-access models. Drawing on a survey of recent work on evaluation and documentation, we identify three persistent shortcomings in current reporting practices: reproducibility, accessibility, and governance. We argue that existing standardization efforts remain insufficient and introduce Evaluation Disclosure Cards (EvalCards) as a path forward. EvalCards are designed to enhance transparency for both researchers and practitioners while providing a practical foundation to meet emerging governance requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_21695 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EvalCards: A Framework for Standardized Evaluation Reporting Dhar, Ruchira Villegas, Danae Sanchez Karamolegkou, Antonia Schiavone, Alice Yuan, Yifei Chen, Xinyi Li, Jiaang Frank, Stella De Grazia, Laura Swain, Monorama Brandl, Stephanie Hershcovich, Daniel Søgaard, Anders Elliott, Desmond Computation and Language Artificial Intelligence Computers and Society Evaluation has long been a central concern in NLP, and transparent reporting practices are more critical than ever in today's landscape of rapidly released open-access models. Drawing on a survey of recent work on evaluation and documentation, we identify three persistent shortcomings in current reporting practices: reproducibility, accessibility, and governance. We argue that existing standardization efforts remain insufficient and introduce Evaluation Disclosure Cards (EvalCards) as a path forward. EvalCards are designed to enhance transparency for both researchers and practitioners while providing a practical foundation to meet emerging governance requirements. |
| title | EvalCards: A Framework for Standardized Evaluation Reporting |
| topic | Computation and Language Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2511.21695 |