Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.03170 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916236489654272 |
|---|---|
| author | Zeng, Yueling Jenny Wang, Li-C. Ibbetson, Thomas |
| author_facet | Zeng, Yueling Jenny Wang, Li-C. Ibbetson, Thomas |
| contents | This work presents a novel approach called oracle-checker scheme for evaluating the answer given by a generative large language model (LLM). Two types of checkers are presented. The first type of checker follows the idea of property testing. The second type of checker follows the idea of program checking. Their applications are demonstrated in two separate contexts, entity extraction and paraphrase decision, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_03170 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Oracle-Checker Scheme for Evaluating a Generative Large Language Model Zeng, Yueling Jenny Wang, Li-C. Ibbetson, Thomas Computation and Language This work presents a novel approach called oracle-checker scheme for evaluating the answer given by a generative large language model (LLM). Two types of checkers are presented. The first type of checker follows the idea of property testing. The second type of checker follows the idea of program checking. Their applications are demonstrated in two separate contexts, entity extraction and paraphrase decision, respectively. |
| title | Oracle-Checker Scheme for Evaluating a Generative Large Language Model |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.03170 |