Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.04947 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913864671559680 |
|---|---|
| author | Li, Yanyang Wong, Tin Long Hung, Cheung To Zhao, Jianqiao Zheng, Duo Liu, Ka Wai Lyu, Michael R. Wang, Liwei |
| author_facet | Li, Yanyang Wong, Tin Long Hung, Cheung To Zhao, Jianqiao Zheng, Duo Liu, Ka Wai Lyu, Michael R. Wang, Liwei |
| contents | Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of access to proprietary training data. To address this issue, we present C$^2$LEVA, a comprehensive bilingual benchmark featuring systematic contamination prevention. C$^2$LEVA firstly offers a holistic evaluation encompassing 22 tasks, each targeting a specific application or ability of LLMs, and secondly a trustworthy assessment due to our contamination-free tasks, ensured by a systematic contamination prevention strategy that fully automates test data renewal and enforces data protection during benchmark data release. Our large-scale evaluation of 15 open-source and proprietary models demonstrates the effectiveness of C$^2$LEVA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04947 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | C$^2$LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation Li, Yanyang Wong, Tin Long Hung, Cheung To Zhao, Jianqiao Zheng, Duo Liu, Ka Wai Lyu, Michael R. Wang, Liwei Computation and Language Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of access to proprietary training data. To address this issue, we present C$^2$LEVA, a comprehensive bilingual benchmark featuring systematic contamination prevention. C$^2$LEVA firstly offers a holistic evaluation encompassing 22 tasks, each targeting a specific application or ability of LLMs, and secondly a trustworthy assessment due to our contamination-free tasks, ensured by a systematic contamination prevention strategy that fully automates test data renewal and enforces data protection during benchmark data release. Our large-scale evaluation of 15 open-source and proprietary models demonstrates the effectiveness of C$^2$LEVA. |
| title | C$^2$LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.04947 |