LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917273378226176 |
|---|---|
| author | Hili, Youssef Attia El Thomas, Albert Tiomoko, Malik Benechehab, Abdelhakim Léger, Corentin Ancourt, Corinne Kégl, Balázs |
| author_facet | Hili, Youssef Attia El Thomas, Albert Tiomoko, Malik Benechehab, Abdelhakim Léger, Corentin Ancourt, Corinne Kégl, Balázs |
| contents | Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we prompt an LLM to recommend both model families and hyperparameters. We study two prompting strategies: (1) a zero-shot mode relying solely on pretrained knowledge, and (2) a meta-informed mode augmented with examples of models and their performance on past tasks. Across synthetic and real-world benchmarks, we show that LLMs can exploit dataset metadata to recommend competitive models and hyperparameters without search, and that improvements from meta-informed prompting demonstrate their capacity for in-context meta-learning. These results highlight a promising new role for LLMs as lightweight, general-purpose assistants for model selection and hyperparameter optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_26510 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection Hili, Youssef Attia El Thomas, Albert Tiomoko, Malik Benechehab, Abdelhakim Léger, Corentin Ancourt, Corinne Kégl, Balázs Machine Learning Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we prompt an LLM to recommend both model families and hyperparameters. We study two prompting strategies: (1) a zero-shot mode relying solely on pretrained knowledge, and (2) a meta-informed mode augmented with examples of models and their performance on past tasks. Across synthetic and real-world benchmarks, we show that LLMs can exploit dataset metadata to recommend competitive models and hyperparameters without search, and that improvements from meta-informed prompting demonstrate their capacity for in-context meta-learning. These results highlight a promising new role for LLMs as lightweight, general-purpose assistants for model selection and hyperparameter optimization. |
| title | LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.26510 |