Improving LLM-based Global Optimization with Search Space Partitioning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Schwanke, Andrej, Ivanov, Lyubomir, Salinas, David, Ferreira, Fabio, Klein, Aaron, Hutter, Frank, Zela, Arber
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866911400359624704
author	Schwanke, Andrej Ivanov, Lyubomir Salinas, David Ferreira, Fabio Klein, Aaron Hutter, Frank Zela, Arber
author_facet	Schwanke, Andrej Ivanov, Lyubomir Salinas, David Ferreira, Fabio Klein, Aaron Hutter, Frank Zela, Arber
contents	Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. Despite promising results, LLM-based methods often struggle in high-dimensional search spaces or when lacking domain-specific priors, leading to sparse or uninformative suggestions. To overcome these limitations, we propose HOLLM, a novel global optimization algorithm that enhances LLM-driven sampling by partitioning the search space into promising subregions. Each subregion acts as a ``meta-arm'' selected via a bandit-inspired scoring mechanism that effectively balances exploration and exploitation. Within each selected subregion, an LLM then proposes high-quality candidate points, without any explicit domain knowledge. Empirical evaluation on standard optimization benchmarks shows that HOLLM consistently matches or surpasses leading global optimization methods, while substantially outperforming global LLM-based sampling strategies.
format	Preprint
id	arxiv_https___arxiv_org_abs_2505_21372
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Improving LLM-based Global Optimization with Search Space Partitioning Schwanke, Andrej Ivanov, Lyubomir Salinas, David Ferreira, Fabio Klein, Aaron Hutter, Frank Zela, Arber Machine Learning Artificial Intelligence Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. Despite promising results, LLM-based methods often struggle in high-dimensional search spaces or when lacking domain-specific priors, leading to sparse or uninformative suggestions. To overcome these limitations, we propose HOLLM, a novel global optimization algorithm that enhances LLM-driven sampling by partitioning the search space into promising subregions. Each subregion acts as a ``meta-arm'' selected via a bandit-inspired scoring mechanism that effectively balances exploration and exploitation. Within each selected subregion, an LLM then proposes high-quality candidate points, without any explicit domain knowledge. Empirical evaluation on standard optimization benchmarks shows that HOLLM consistently matches or surpasses leading global optimization methods, while substantially outperforming global LLM-based sampling strategies.
title	Improving LLM-based Global Optimization with Search Space Partitioning
topic	Machine Learning Artificial Intelligence
url	https://arxiv.org/abs/2505.21372