Multi-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909988821139456 |
|---|---|
| author | Zhang, Chengqian Zhang, Duo Peng, Anyang Guo, Mingyu Zhang, Yuzhi Wang, Lei Ke, Guolin Zhang, Linfeng Li, Tiejun Wang, Han |
| author_facet | Zhang, Chengqian Zhang, Duo Peng, Anyang Guo, Mingyu Zhang, Yuzhi Wang, Lei Ke, Guolin Zhang, Linfeng Li, Tiejun Wang, Han |
| contents | Accurate de novo molecular and materials design requires structure-property models that generalize beyond known regimes. Although pretrained atomistic models achieve strong in-distribution accuracy after fine-tuning, their reliability under out-of-distribution (OOD) conditions remains unclear. We identify a critical failure mode in downstream adaptation: standard fine-tuning induces representation collapse, erasing pretrained chemical and structural priors and severely degrading OOD performance. To address this limitation, we propose multi-task fine-tuning (MFT), which jointly optimizes downstream property prediction with a physically grounded force-field objective inherited from pretraining. This approach preserves essential chemical priors while enabling task-specific adaptation. Across molecular and materials benchmarks, MFT consistently improves OOD generalization, approaching the theoretical limit set by in-distribution accuracy, while outperforming standard fine-tuning, training from scratch, and state-of-the-art task-specific models. These results establish safe adaptation as a central requirement for large atomistic models and position MFT as a practical and data-efficient pathway toward robust molecular and materials discovery. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_08486 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multi-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models Zhang, Chengqian Zhang, Duo Peng, Anyang Guo, Mingyu Zhang, Yuzhi Wang, Lei Ke, Guolin Zhang, Linfeng Li, Tiejun Wang, Han Computational Physics Accurate de novo molecular and materials design requires structure-property models that generalize beyond known regimes. Although pretrained atomistic models achieve strong in-distribution accuracy after fine-tuning, their reliability under out-of-distribution (OOD) conditions remains unclear. We identify a critical failure mode in downstream adaptation: standard fine-tuning induces representation collapse, erasing pretrained chemical and structural priors and severely degrading OOD performance. To address this limitation, we propose multi-task fine-tuning (MFT), which jointly optimizes downstream property prediction with a physically grounded force-field objective inherited from pretraining. This approach preserves essential chemical priors while enabling task-specific adaptation. Across molecular and materials benchmarks, MFT consistently improves OOD generalization, approaching the theoretical limit set by in-distribution accuracy, while outperforming standard fine-tuning, training from scratch, and state-of-the-art task-specific models. These results establish safe adaptation as a central requirement for large atomistic models and position MFT as a practical and data-efficient pathway toward robust molecular and materials discovery. |
| title | Multi-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models |
| topic | Computational Physics |
| url | https://arxiv.org/abs/2601.08486 |