Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2508.03716 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917299448971264 |
|---|---|
| author | Richmond, Paul Agarwal, Prarit Chowdhury, Borun Niarchos, Vasilis Papageorgakis, Constantinos |
| author_facet | Richmond, Paul Agarwal, Prarit Chowdhury, Borun Niarchos, Vasilis Papageorgakis, Constantinos |
| contents | We present specialized Large Language Models for theoretical High-Energy Physics, obtained as 20 fine-tuned variants of the 8-billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also trained models on datasets that contained abstracts from disparate fields such as the q-bio and cs categories. All models were fine-tuned using two distinct Low-Rank Adaptation fine-tuning approaches and varying dataset sizes, and outperformed the base model on hep-th abstract completion tasks. We compare performance against leading commercial LLMs (ChatGPT, Claude, Gemini, DeepSeek) and derive insights for further developing specialized language models for High-Energy Theoretical Physics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03716 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FeynTune: Large Language Models for High-Energy Theory Richmond, Paul Agarwal, Prarit Chowdhury, Borun Niarchos, Vasilis Papageorgakis, Constantinos Computation and Language Machine Learning High Energy Physics - Theory We present specialized Large Language Models for theoretical High-Energy Physics, obtained as 20 fine-tuned variants of the 8-billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also trained models on datasets that contained abstracts from disparate fields such as the q-bio and cs categories. All models were fine-tuned using two distinct Low-Rank Adaptation fine-tuning approaches and varying dataset sizes, and outperformed the base model on hep-th abstract completion tasks. We compare performance against leading commercial LLMs (ChatGPT, Claude, Gemini, DeepSeek) and derive insights for further developing specialized language models for High-Energy Theoretical Physics. |
| title | FeynTune: Large Language Models for High-Energy Theory |
| topic | Computation and Language Machine Learning High Energy Physics - Theory |
| url | https://arxiv.org/abs/2508.03716 |