Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916521463250944 |
|---|---|
| author | Savage, Thomas Ma, Stephen Boukil, Abdessalem Patel, Vishwesh Rangan, Ekanath Lopez, Ivan Chen, Jonathan H |
| author_facet | Savage, Thomas Ma, Stephen Boukil, Abdessalem Patel, Vishwesh Rangan, Ekanath Lopez, Ivan Chen, Jonathan H |
| contents | Large Language Model (LLM) fine tuning is underutilized in the field of medicine. Two of the most common methods of fine tuning are Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO), but there is little guidance informing users when to use either technique. In this investigation, we compare the performance of SFT and DPO for five common natural language tasks in medicine: Classification with text data, Classification with numeric data, Clinical Reasoning, Summarization, and Clinical Triage. We find that SFT alone is sufficient for Classification with text data, whereas DPO improves performance for the more complex tasks of Clinical Reasoning, Summarization and Clinical Triage. Our results establish the role and importance of DPO fine tuning within medicine, and consequently call attention to current software gaps that prevent widespread deployment of this technique. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12741 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization Savage, Thomas Ma, Stephen Boukil, Abdessalem Patel, Vishwesh Rangan, Ekanath Lopez, Ivan Chen, Jonathan H Computation and Language Artificial Intelligence Large Language Model (LLM) fine tuning is underutilized in the field of medicine. Two of the most common methods of fine tuning are Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO), but there is little guidance informing users when to use either technique. In this investigation, we compare the performance of SFT and DPO for five common natural language tasks in medicine: Classification with text data, Classification with numeric data, Clinical Reasoning, Summarization, and Clinical Triage. We find that SFT alone is sufficient for Classification with text data, whereas DPO improves performance for the more complex tasks of Clinical Reasoning, Summarization and Clinical Triage. Our results establish the role and importance of DPO fine tuning within medicine, and consequently call attention to current software gaps that prevent widespread deployment of this technique. |
| title | Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.12741 |