Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910511150399488 |
|---|---|
| author | Liu, Ning Jafarzadeh, Siavash Lattimer, Brian Y. Ni, Shuna Lua, Jim Yu, Yue |
| author_facet | Liu, Ning Jafarzadeh, Siavash Lattimer, Brian Y. Ni, Shuna Lua, Jim Yu, Yue |
| contents | Large language models (LLMs) bear promise as a fast and accurate material modeling paradigm for evaluation, analysis, and design. Their vast number of trainable parameters necessitates a wealth of data to achieve accuracy and mitigate overfitting. However, experimental measurements are often limited and costly to obtain in sufficient quantities for finetuning. To this end, we present a physics-based training pipeline that tackles the pathology of data scarcity. The core enabler is a physics-based modeling framework that generates a multitude of synthetic data to align the LLM to a physically consistent initial state before finetuning. Our framework features a two-phase training strategy: (1) utilizing the large-in-amount while less accurate synthetic data for supervised pretraining, and (2) finetuning the phase-1 model with limited experimental data. We empirically demonstrate that supervised pretraining is vital to obtaining accurate finetuned LLMs, via the lens of learning polymer flammability metrics where cone calorimeter data is sparse. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02770 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties Liu, Ning Jafarzadeh, Siavash Lattimer, Brian Y. Ni, Shuna Lua, Jim Yu, Yue Machine Learning Computational Engineering, Finance, and Science Large language models (LLMs) bear promise as a fast and accurate material modeling paradigm for evaluation, analysis, and design. Their vast number of trainable parameters necessitates a wealth of data to achieve accuracy and mitigate overfitting. However, experimental measurements are often limited and costly to obtain in sufficient quantities for finetuning. To this end, we present a physics-based training pipeline that tackles the pathology of data scarcity. The core enabler is a physics-based modeling framework that generates a multitude of synthetic data to align the LLM to a physically consistent initial state before finetuning. Our framework features a two-phase training strategy: (1) utilizing the large-in-amount while less accurate synthetic data for supervised pretraining, and (2) finetuning the phase-1 model with limited experimental data. We empirically demonstrate that supervised pretraining is vital to obtaining accurate finetuned LLMs, via the lens of learning polymer flammability metrics where cone calorimeter data is sparse. |
| title | Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties |
| topic | Machine Learning Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2407.02770 |