Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.04310 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908498010308608 |
|---|---|
| author | Sun, Weiwei Feng, Shengyu Li, Shanda Yang, Yiming |
| author_facet | Sun, Weiwei Feng, Shengyu Li, Shanda Yang, Yiming |
| contents | Although LLM-based agents have attracted significant attention in domains such as software engineering and machine learning research, their role in advancing combinatorial optimization (CO) remains relatively underexplored. This gap underscores the need for a deeper understanding of their potential in tackling structured, constraint-intensive problems -- a pursuit currently limited by the absence of comprehensive benchmarks for systematic investigation. To address this, we introduce CO-Bench, a benchmark suite featuring 36 real-world CO problems drawn from a broad range of domains and complexity levels. CO-Bench includes structured problem formulations and curated data to support rigorous investigation of LLM agents. We evaluate multiple agentic frameworks against established human-designed algorithms, revealing the strengths and limitations of existing LLM agents and identifying promising directions for future research. CO-Bench is publicly available at https://github.com/sunnweiwei/CO-Bench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04310 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CO-Bench: Benchmarking Language Model Agents in Algorithm Search for Combinatorial Optimization Sun, Weiwei Feng, Shengyu Li, Shanda Yang, Yiming Computation and Language Artificial Intelligence Although LLM-based agents have attracted significant attention in domains such as software engineering and machine learning research, their role in advancing combinatorial optimization (CO) remains relatively underexplored. This gap underscores the need for a deeper understanding of their potential in tackling structured, constraint-intensive problems -- a pursuit currently limited by the absence of comprehensive benchmarks for systematic investigation. To address this, we introduce CO-Bench, a benchmark suite featuring 36 real-world CO problems drawn from a broad range of domains and complexity levels. CO-Bench includes structured problem formulations and curated data to support rigorous investigation of LLM agents. We evaluate multiple agentic frameworks against established human-designed algorithms, revealing the strengths and limitations of existing LLM agents and identifying promising directions for future research. CO-Bench is publicly available at https://github.com/sunnweiwei/CO-Bench. |
| title | CO-Bench: Benchmarking Language Model Agents in Algorithm Search for Combinatorial Optimization |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.04310 |