Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908515649454080 |
|---|---|
| author | Stoisser, Josefa Lia Martell, Marc Boubnovski Phillips, Lawrence Mazzoni, Gianluca Harder, Lea Mørch Torr, Philip Ferkinghoff-Borg, Jesper Martens, Kaspar Fauqueur, Julien |
| author_facet | Stoisser, Josefa Lia Martell, Marc Boubnovski Phillips, Lawrence Mazzoni, Gianluca Harder, Lea Mørch Torr, Philip Ferkinghoff-Borg, Jesper Martens, Kaspar Fauqueur, Julien |
| contents | Large language model (LLM) agents are increasingly deployed in structured biomedical data environments, yet they often produce fluent but overconfident outputs when reasoning over complex multi-table data. We introduce an uncertainty-aware agent for query-conditioned multi-table summarization that leverages two complementary signals: (i) retrieval uncertainty--entropy over multiple table-selection rollouts--and (ii) summary uncertainty--combining self-consistency and perplexity. Summary uncertainty is incorporated into reinforcement learning (RL) with Group Relative Policy Optimization (GRPO), while both retrieval and summary uncertainty guide inference-time filtering and support the construction of higher-quality synthetic datasets.
On multi-omics benchmarks, our approach improves factuality and calibration, nearly tripling correct and useful claims per summary (3.0\(\rightarrow\)8.4 internal; 3.6\(\rightarrow\)9.9 cancer multi-omics) and substantially improving downstream survival prediction (C-index 0.32\(\rightarrow\)0.63). These results demonstrate that uncertainty can serve as a control signal--enabling agents to abstain, communicate confidence, and become more reliable tools for complex structured-data environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_02401 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning Stoisser, Josefa Lia Martell, Marc Boubnovski Phillips, Lawrence Mazzoni, Gianluca Harder, Lea Mørch Torr, Philip Ferkinghoff-Borg, Jesper Martens, Kaspar Fauqueur, Julien Artificial Intelligence Large language model (LLM) agents are increasingly deployed in structured biomedical data environments, yet they often produce fluent but overconfident outputs when reasoning over complex multi-table data. We introduce an uncertainty-aware agent for query-conditioned multi-table summarization that leverages two complementary signals: (i) retrieval uncertainty--entropy over multiple table-selection rollouts--and (ii) summary uncertainty--combining self-consistency and perplexity. Summary uncertainty is incorporated into reinforcement learning (RL) with Group Relative Policy Optimization (GRPO), while both retrieval and summary uncertainty guide inference-time filtering and support the construction of higher-quality synthetic datasets. On multi-omics benchmarks, our approach improves factuality and calibration, nearly tripling correct and useful claims per summary (3.0\(\rightarrow\)8.4 internal; 3.6\(\rightarrow\)9.9 cancer multi-omics) and substantially improving downstream survival prediction (C-index 0.32\(\rightarrow\)0.63). These results demonstrate that uncertainty can serve as a control signal--enabling agents to abstain, communicate confidence, and become more reliable tools for complex structured-data environments. |
| title | Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.02401 |