Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.00122 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909275337523200 |
|---|---|
| author | Hou, Yufang Tran, Thy Thy Vu, Doan Nam Long Cao, Yiwen Li, Kai Rohde, Lukas Gurevych, Iryna |
| author_facet | Hou, Yufang Tran, Thy Thy Vu, Doan Nam Long Cao, Yiwen Li, Kai Rohde, Lukas Gurevych, Iryna |
| contents | This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP) and designing course assignments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00122 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Course Shared Task on Evaluating LLM Output for Clinical Questions Hou, Yufang Tran, Thy Thy Vu, Doan Nam Long Cao, Yiwen Li, Kai Rohde, Lukas Gurevych, Iryna Computation and Language This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP) and designing course assignments. |
| title | A Course Shared Task on Evaluating LLM Output for Clinical Questions |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.00122 |