A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914865260527616 |
|---|---|
| author | Tan, Ting Fang Elangovan, Kabilan Ong, Jasmine Shah, Nigam Sung, Joseph Wong, Tien Yin Xue, Lan Liu, Nan Wang, Haibo Kuo, Chang Fu Chesterman, Simon Yeong, Zee Kin Ting, Daniel SW |
| author_facet | Tan, Ting Fang Elangovan, Kabilan Ong, Jasmine Shah, Nigam Sung, Joseph Wong, Tien Yin Xue, Lan Liu, Nan Wang, Haibo Kuo, Chang Fu Chesterman, Simon Yeong, Zee Kin Ting, Daniel SW |
| contents | A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare that expands beyond traditional accuracy and quantitative metrics needed. We propose 5 key aspects for evaluation of LLMs: Safety, Consensus, Objectivity, Reproducibility and Explainability (S.C.O.R.E.). We suggest that S.C.O.R.E. may form the basis for an evaluation framework for future LLM-based models that are safe, reliable, trustworthy, and ethical for healthcare and clinical applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_07666 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability Tan, Ting Fang Elangovan, Kabilan Ong, Jasmine Shah, Nigam Sung, Joseph Wong, Tien Yin Xue, Lan Liu, Nan Wang, Haibo Kuo, Chang Fu Chesterman, Simon Yeong, Zee Kin Ting, Daniel SW Computation and Language Artificial Intelligence A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare that expands beyond traditional accuracy and quantitative metrics needed. We propose 5 key aspects for evaluation of LLMs: Safety, Consensus, Objectivity, Reproducibility and Explainability (S.C.O.R.E.). We suggest that S.C.O.R.E. may form the basis for an evaluation framework for future LLM-based models that are safe, reliable, trustworthy, and ethical for healthcare and clinical applications. |
| title | A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.07666 |