Discovering and Learning Probabilistic Models of Black-Box AI Capabilities
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908724574027776 |
|---|---|
| author | Bramblett, Daniel Karia, Rushang Ciotinga, Adrian Suresh, Ruthvick Verma, Pulkit Choi, YooJung Srivastava, Siddharth |
| author_facet | Bramblett, Daniel Karia, Rushang Ciotinga, Adrian Suresh, Ruthvick Verma, Pulkit Choi, YooJung Srivastava, Siddharth |
| contents | Black-box AI (BBAI) systems such as foundational models are increasingly being used for sequential decision making. To ensure that such systems are safe to operate and deploy, it is imperative to develop efficient methods that can provide a sound and interpretable representation of the BBAI's capabilities. This paper shows that PDDL-style representations can be used to efficiently learn and model an input BBAI's planning capabilities. It uses the Monte-Carlo tree search paradigm to systematically create test tasks, acquire data, and prune the hypothesis space of possible symbolic models. Learned models describe a BBAI's capabilities, the conditions under which they can be executed, and the possible outcomes of executing them along with their associated probabilities. Theoretical results show soundness, completeness and convergence of the learned models. Empirical results with multiple BBAI systems illustrate the scope, efficiency, and accuracy of the presented methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16733 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Discovering and Learning Probabilistic Models of Black-Box AI Capabilities Bramblett, Daniel Karia, Rushang Ciotinga, Adrian Suresh, Ruthvick Verma, Pulkit Choi, YooJung Srivastava, Siddharth Artificial Intelligence Black-box AI (BBAI) systems such as foundational models are increasingly being used for sequential decision making. To ensure that such systems are safe to operate and deploy, it is imperative to develop efficient methods that can provide a sound and interpretable representation of the BBAI's capabilities. This paper shows that PDDL-style representations can be used to efficiently learn and model an input BBAI's planning capabilities. It uses the Monte-Carlo tree search paradigm to systematically create test tasks, acquire data, and prune the hypothesis space of possible symbolic models. Learned models describe a BBAI's capabilities, the conditions under which they can be executed, and the possible outcomes of executing them along with their associated probabilities. Theoretical results show soundness, completeness and convergence of the learned models. Empirical results with multiple BBAI systems illustrate the scope, efficiency, and accuracy of the presented methods. |
| title | Discovering and Learning Probabilistic Models of Black-Box AI Capabilities |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.16733 |