Guiding LLM Temporal Logic Generation with Explicit Separation of Data and Control
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914831775301632 |
|---|---|
| author | Murphy, William Holzer, Nikolaus Koenig, Nathan Cui, Leyi Rothkopf, Raven Qiao, Feitong Santolucito, Mark |
| author_facet | Murphy, William Holzer, Nikolaus Koenig, Nathan Cui, Leyi Rothkopf, Raven Qiao, Feitong Santolucito, Mark |
| contents | Temporal logics are powerful tools that are widely used for the synthesis and verification of reactive systems. The recent progress on Large Language Models (LLMs) has the potential to make the process of writing such specifications more accessible. However, writing specifications in temporal logics remains challenging for all but the most expert users. A key question in using LLMs for temporal logic specification engineering is to understand what kind of guidance is most helpful to the LLM and the users to easily produce specifications. Looking specifically at the problem of reactive program synthesis, we explore the impact of providing an LLM with guidance on the separation of control and data--making explicit for the LLM what functionality is relevant for the specification, and treating the remaining functionality as an implementation detail for a series of pre-defined functions and predicates. We present a benchmark set and find that this separation of concerns improves specification generation. Our benchmark provides a test set against which to verify future work in LLM generation of temporal logic specifications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07400 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Guiding LLM Temporal Logic Generation with Explicit Separation of Data and Control Murphy, William Holzer, Nikolaus Koenig, Nathan Cui, Leyi Rothkopf, Raven Qiao, Feitong Santolucito, Mark Machine Learning Logic in Computer Science Temporal logics are powerful tools that are widely used for the synthesis and verification of reactive systems. The recent progress on Large Language Models (LLMs) has the potential to make the process of writing such specifications more accessible. However, writing specifications in temporal logics remains challenging for all but the most expert users. A key question in using LLMs for temporal logic specification engineering is to understand what kind of guidance is most helpful to the LLM and the users to easily produce specifications. Looking specifically at the problem of reactive program synthesis, we explore the impact of providing an LLM with guidance on the separation of control and data--making explicit for the LLM what functionality is relevant for the specification, and treating the remaining functionality as an implementation detail for a series of pre-defined functions and predicates. We present a benchmark set and find that this separation of concerns improves specification generation. Our benchmark provides a test set against which to verify future work in LLM generation of temporal logic specifications. |
| title | Guiding LLM Temporal Logic Generation with Explicit Separation of Data and Control |
| topic | Machine Learning Logic in Computer Science |
| url | https://arxiv.org/abs/2406.07400 |