ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911485230317568 |
|---|---|
| author | Ragsdale, Jarrod Boppana, Rajendra |
| author_facet | Ragsdale, Jarrod Boppana, Rajendra |
| contents | Modeling of command-line interface (CLI) interaction has enabled flexible, execution-free output presentation. However, current approaches struggle to model inputs with complex compositions and inputs whose execution behavior depends on system characteristics. This is due to a lack of shell input-output (ShIO) data in the training distributions used by the models in these approaches. To address this data gap, we present ShIOEnv, a Gymnasium-compatible Bash shell environment for command synthesis and system-grounded execution behavior capturing. To concentrate synthesis on productive regions of the state-action space, we temporally abstract argument construction into grammar-derived options, thereby constraining synthesis to syntactically valid arguments. We introduce a self-supervised irreducibility signal to approximate the proportion of arguments that contribute to the observed execution behavior, serving as a measure of information density for each input. Using ShIOEnv, we curate and release 2.1M input-output pairs for modeling feedback from Bash command execution. We find that models trained on grammar-constrained datasets with higher maximum irreducibility achieve greater accuracy when modeling the execution behavior of user-sourced inputs than prior execution-free baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18374 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling Ragsdale, Jarrod Boppana, Rajendra Computation and Language Artificial Intelligence Machine Learning Modeling of command-line interface (CLI) interaction has enabled flexible, execution-free output presentation. However, current approaches struggle to model inputs with complex compositions and inputs whose execution behavior depends on system characteristics. This is due to a lack of shell input-output (ShIO) data in the training distributions used by the models in these approaches. To address this data gap, we present ShIOEnv, a Gymnasium-compatible Bash shell environment for command synthesis and system-grounded execution behavior capturing. To concentrate synthesis on productive regions of the state-action space, we temporally abstract argument construction into grammar-derived options, thereby constraining synthesis to syntactically valid arguments. We introduce a self-supervised irreducibility signal to approximate the proportion of arguments that contribute to the observed execution behavior, serving as a measure of information density for each input. Using ShIOEnv, we curate and release 2.1M input-output pairs for modeling feedback from Bash command execution. We find that models trained on grammar-constrained datasets with higher maximum irreducibility achieve greater accuracy when modeling the execution behavior of user-sourced inputs than prior execution-free baselines. |
| title | ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2505.18374 |