Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.19487 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912556096946176 |
|---|---|
| author | Ying, Wangyang Zhang, Jinghan Bai, Haoyue Gong, Nanxu Wang, Xinyuan Liu, Kunpeng Reddy, Chandan K. Fu, Yanjie |
| author_facet | Ying, Wangyang Zhang, Jinghan Bai, Haoyue Gong, Nanxu Wang, Xinyuan Liu, Kunpeng Reddy, Chandan K. Fu, Yanjie |
| contents | Discovering interpretable mathematical equations from observed data (a.k.a. equation discovery or symbolic regression) is a cornerstone of scientific discovery, enabling transparent modeling of physical, biological, and economic systems. While foundation models pre-trained on large-scale equation datasets offer a promising starting point, they often suffer from negative transfer and poor generalization when applied to small, domain-specific datasets. In this paper, we introduce EQUATE (Equation Generation via QUality-Aligned Transfer Embeddings), a data-efficient fine-tuning framework that adapts foundation models for symbolic equation discovery in low-data regimes via distillation. EQUATE combines symbolic-numeric alignment with evaluator-guided embedding optimization, enabling a principled embedding-search-generation paradigm. Our approach reformulates discrete equation search as a continuous optimization task in a shared embedding space, guided by data-equation fitness and simplicity. Experiments across three standard public benchmarks (Feynman, Strogatz, and black-box datasets) demonstrate that EQUATE consistently outperforms state-of-the-art baselines in both accuracy and robustness, while preserving low complexity and fast inference. These results highlight EQUATE as a practical and generalizable solution for data-efficient symbolic regression in foundation model distillation settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_19487 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Data-Efficient Symbolic Regression via Foundation Model Distillation Ying, Wangyang Zhang, Jinghan Bai, Haoyue Gong, Nanxu Wang, Xinyuan Liu, Kunpeng Reddy, Chandan K. Fu, Yanjie Machine Learning Artificial Intelligence Discovering interpretable mathematical equations from observed data (a.k.a. equation discovery or symbolic regression) is a cornerstone of scientific discovery, enabling transparent modeling of physical, biological, and economic systems. While foundation models pre-trained on large-scale equation datasets offer a promising starting point, they often suffer from negative transfer and poor generalization when applied to small, domain-specific datasets. In this paper, we introduce EQUATE (Equation Generation via QUality-Aligned Transfer Embeddings), a data-efficient fine-tuning framework that adapts foundation models for symbolic equation discovery in low-data regimes via distillation. EQUATE combines symbolic-numeric alignment with evaluator-guided embedding optimization, enabling a principled embedding-search-generation paradigm. Our approach reformulates discrete equation search as a continuous optimization task in a shared embedding space, guided by data-equation fitness and simplicity. Experiments across three standard public benchmarks (Feynman, Strogatz, and black-box datasets) demonstrate that EQUATE consistently outperforms state-of-the-art baselines in both accuracy and robustness, while preserving low complexity and fast inference. These results highlight EQUATE as a practical and generalizable solution for data-efficient symbolic regression in foundation model distillation settings. |
| title | Data-Efficient Symbolic Regression via Foundation Model Distillation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.19487 |