On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913535388286976 |
|---|---|
| author | Siddiqui, Shoaib Ahmed Chen, Yanzhi Heo, Juyeon Xia, Menglin Weller, Adrian |
| author_facet | Siddiqui, Shoaib Ahmed Chen, Yanzhi Heo, Juyeon Xia, Menglin Weller, Adrian |
| contents | Recent works have successfully applied Large Language Models (LLMs) to function modeling tasks. However, the reasons behind this success remain unclear. In this work, we propose a new evaluation framework to comprehensively assess LLMs' function modeling abilities. By adopting a Bayesian perspective of function modeling, we discover that LLMs are relatively weak in understanding patterns in raw data, but excel at utilizing prior knowledge about the domain to develop a strong understanding of the underlying function. Our findings offer new insights about the strengths and limitations of LLMs in the context of function modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_04541 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective Siddiqui, Shoaib Ahmed Chen, Yanzhi Heo, Juyeon Xia, Menglin Weller, Adrian Machine Learning Artificial Intelligence Recent works have successfully applied Large Language Models (LLMs) to function modeling tasks. However, the reasons behind this success remain unclear. In this work, we propose a new evaluation framework to comprehensively assess LLMs' function modeling abilities. By adopting a Bayesian perspective of function modeling, we discover that LLMs are relatively weak in understanding patterns in raw data, but excel at utilizing prior knowledge about the domain to develop a strong understanding of the underlying function. Our findings offer new insights about the strengths and limitations of LLMs in the context of function modeling. |
| title | On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.04541 |