Code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code"
Fuente:
Zenodo
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866902320258744320 |
|---|---|
| author | Corrêa, Augusto B. Pereira, André Grahl Seipp, Jendrik |
| author_facet | Corrêa, Augusto B. Pereira, André Grahl Seipp, Jendrik |
| contents | <p>Source code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code"</p> <p><strong>Installing Dependencies</strong></p> <p>You'll need <a href="https://docs.astral.sh/uv/getting-started/installation/" target="_blank" rel="noopener">uv</a> installed on your system. Then you can run: <code>uv sync</code></p> <p><strong> API Keys</strong></p> <p>We support several frameworks (i.e., APIs to communicate with the LLMs). For each API, you need to setup its related environment variable with a valid API key:</p> <p>- for the Google Gemini API you must define GOOGLE_API_KEY<br>- for the DeepSeek API you must define DEEPSEEK_API_KEY<br>- for the OpenAI API you must define OPENAI_API_KEY</p> <p><strong>Heuristic Generation</strong></p> <p>The basic command is:</p> <p><code>uv run llm-heuristics.py --domain DOMAIN</code><br><br>where `DOMAIN` is the name of the domain for which the heuristic is generated. The domains used in the paper are blocksworld, childsnack, floortile, miconic, rovers, sokoban, spanner, and transport.</p> <p>There are several other options you can pass to the command line:</p> <p>- <code>--domain</code>: choose the domain that you want to use.<br>- <code>--framework</code>: choose what framework you want to use. Available choices are: `gemini` (default), `deepseek`, and `openai`.<br>- <code>--model:</code> choose the LLM model to be used. The available models depend on the selected framework.<br>- <code>--prompt-format</code>: choose prompt format. *Note: this option was used during development and is no longer necessary; it will be removed in the future.* (Default: "neurips")<br>- <code>--heuristic-name</code>: name of the heuristic and of its class.<br>-<code> --heuristic-file</code>: file where the learned heuristic is stored. It must end with `.py`.<br>- <code>--temperature:</code> temperature (default: 1.0)<br>- <code>--top-p: </code>top-k value (default: 0.5)<br>- <code>--ablation</code>: choose a component to do the ablation. Omit this option if you want the complete prompt.</p> <p><strong>Pyperplan</strong></p> <p>To run Pyperplan with your generated heuristic, execute</p> <p><code>uv run src/pyperplan/pyperplan.py -s gbfs_early_goal_test -H HEURISTIC-FILE /path/to/domain.pddl /path/to/instance.pddl</code></p> <p>where HEURISTIC-FILE is the (relative or absolute) path to the heuristic generated in the previous step.</p> <p>Alternatively, you can use hff to run the FF heuristic or blind to run the blind heuristic (effectively, no heuristic).</p> <p> <strong>Example</strong></p> <p>Let's say you want to learn a heuristic for the `blocksworld` domain with the name `AmazingHeuristic`. Then you should run</p> <p><code>uv run llm-heuristics.py --domain blocksworld --heuristic-name AmazingHeuristic --heuristic-file amazing-heuristic.py</code></p> <p><br>Now we can call Pyperplan using the new heuristic with the following command to solve instance<br>an instance of the `blocksworld` testing set:</p> <p><code>uv run src/pyperplan/pyperplan.py -H amazing-heuristic.py -s gbfs_early_goal_test benchmarks/ipc2023-learning/testing/blocksworld/easy-p03.pddl</code></p> <p><strong>End-to-End Plan Generation</strong></p> <p>The basic command is:</p> <p><code>uv run end-to-end.py --domain DOMAIN --instance INSTANCE</code><br><br>where `DOMAIN` is the name of the domain of the instance, and `INSTANCE` is the particular instance for which we are computing a plan.</p> <p>Other available options are:<br>- --framework: choose LLM framework. Available choices are: `gemini` (default), `deepseek`, and `openai`.<br>- --model: choose the LLM model. The available models depend on the selected framework.<br>- --plan-file: file where the computed heuristic is stored.<br>- --temperature: temperature (default: 0.1)<br>- --top-p: top-k value (default: 0.5)</p> <p><strong>Experimental Data</strong></p> <p>We also include the relevant experimental data for the main contributions of the paper. These are the directories containing experimental data used in the paper:</p> <p>- archived-heuristics: Contains all heuristics generated by the models used, including the ones for the ablation study.<br>- logs: Contains all logs from the heuristic generation phase. It contains the prompt, the answer, and the code extracted from the answer.<br>- benchmarks: Contains all the domains from the Learning Track of the IPC 2023 benchmark used in the paper. The benchmark set is split into training and testing sets.<br>- experiments: Contains the scripts used for the planning experiments (i.e., Pyperplan experiments). The subdirectory `data` contains the HTML reports summarizing all experiments.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17400964 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code" Corrêa, Augusto B. Pereira, André Grahl Seipp, Jendrik Classical Planning Heuristic Search Large-Language Models <p>Source code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code"</p> <p><strong>Installing Dependencies</strong></p> <p>You'll need <a href="https://docs.astral.sh/uv/getting-started/installation/" target="_blank" rel="noopener">uv</a> installed on your system. Then you can run: <code>uv sync</code></p> <p><strong> API Keys</strong></p> <p>We support several frameworks (i.e., APIs to communicate with the LLMs). For each API, you need to setup its related environment variable with a valid API key:</p> <p>- for the Google Gemini API you must define GOOGLE_API_KEY<br>- for the DeepSeek API you must define DEEPSEEK_API_KEY<br>- for the OpenAI API you must define OPENAI_API_KEY</p> <p><strong>Heuristic Generation</strong></p> <p>The basic command is:</p> <p><code>uv run llm-heuristics.py --domain DOMAIN</code><br><br>where `DOMAIN` is the name of the domain for which the heuristic is generated. The domains used in the paper are blocksworld, childsnack, floortile, miconic, rovers, sokoban, spanner, and transport.</p> <p>There are several other options you can pass to the command line:</p> <p>- <code>--domain</code>: choose the domain that you want to use.<br>- <code>--framework</code>: choose what framework you want to use. Available choices are: `gemini` (default), `deepseek`, and `openai`.<br>- <code>--model:</code> choose the LLM model to be used. The available models depend on the selected framework.<br>- <code>--prompt-format</code>: choose prompt format. *Note: this option was used during development and is no longer necessary; it will be removed in the future.* (Default: "neurips")<br>- <code>--heuristic-name</code>: name of the heuristic and of its class.<br>-<code> --heuristic-file</code>: file where the learned heuristic is stored. It must end with `.py`.<br>- <code>--temperature:</code> temperature (default: 1.0)<br>- <code>--top-p: </code>top-k value (default: 0.5)<br>- <code>--ablation</code>: choose a component to do the ablation. Omit this option if you want the complete prompt.</p> <p><strong>Pyperplan</strong></p> <p>To run Pyperplan with your generated heuristic, execute</p> <p><code>uv run src/pyperplan/pyperplan.py -s gbfs_early_goal_test -H HEURISTIC-FILE /path/to/domain.pddl /path/to/instance.pddl</code></p> <p>where HEURISTIC-FILE is the (relative or absolute) path to the heuristic generated in the previous step.</p> <p>Alternatively, you can use hff to run the FF heuristic or blind to run the blind heuristic (effectively, no heuristic).</p> <p> <strong>Example</strong></p> <p>Let's say you want to learn a heuristic for the `blocksworld` domain with the name `AmazingHeuristic`. Then you should run</p> <p><code>uv run llm-heuristics.py --domain blocksworld --heuristic-name AmazingHeuristic --heuristic-file amazing-heuristic.py</code></p> <p><br>Now we can call Pyperplan using the new heuristic with the following command to solve instance<br>an instance of the `blocksworld` testing set:</p> <p><code>uv run src/pyperplan/pyperplan.py -H amazing-heuristic.py -s gbfs_early_goal_test benchmarks/ipc2023-learning/testing/blocksworld/easy-p03.pddl</code></p> <p><strong>End-to-End Plan Generation</strong></p> <p>The basic command is:</p> <p><code>uv run end-to-end.py --domain DOMAIN --instance INSTANCE</code><br><br>where `DOMAIN` is the name of the domain of the instance, and `INSTANCE` is the particular instance for which we are computing a plan.</p> <p>Other available options are:<br>- --framework: choose LLM framework. Available choices are: `gemini` (default), `deepseek`, and `openai`.<br>- --model: choose the LLM model. The available models depend on the selected framework.<br>- --plan-file: file where the computed heuristic is stored.<br>- --temperature: temperature (default: 0.1)<br>- --top-p: top-k value (default: 0.5)</p> <p><strong>Experimental Data</strong></p> <p>We also include the relevant experimental data for the main contributions of the paper. These are the directories containing experimental data used in the paper:</p> <p>- archived-heuristics: Contains all heuristics generated by the models used, including the ones for the ablation study.<br>- logs: Contains all logs from the heuristic generation phase. It contains the prompt, the answer, and the code extracted from the answer.<br>- benchmarks: Contains all the domains from the Learning Track of the IPC 2023 benchmark used in the paper. The benchmark set is split into training and testing sets.<br>- experiments: Contains the scripts used for the planning experiments (i.e., Pyperplan experiments). The subdirectory `data` contains the HTML reports summarizing all experiments.</p> |
| title | Code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code" |
| topic | Classical Planning Heuristic Search Large-Language Models |
| url | https://doi.org/10.5281/zenodo.17400964 |