Copertina

Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Stoisser, Josefa Lia, Martell, Marc Boubnovski, Boldsen, Sidsel, Märtens, Kaspar, Kitchen, Robert
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Measuring Black-Box Confidence via Reasoning Trajectories: Geometry, Coverage, and Verbalization
di: Martell, Marc Boubnovski, et al.
Pubblicazione: (2026)

Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning
di: Stoisser, Josefa Lia, et al.
Pubblicazione: (2025)

MechPert: Mechanistic Consensus as an Inductive Bias for Unseen Perturbation Prediction
di: Martell, Marc Boubnovski, et al.
Pubblicazione: (2026)

STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsing
di: Stoisser, Josefa Lia, et al.
Pubblicazione: (2025)

Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning
di: Stoisser, Josefa Lia, et al.
Pubblicazione: (2025)

Towards Label-Free Biological Reasoning Synthetic Dataset Creation via Uncertainty Filtering
di: Stoisser, Josefa Lia, et al.
Pubblicazione: (2025)

SynthPert: Enhancing LLM Biological Reasoning via Synthetic Reasoning Traces for Cellular Perturbation Prediction
di: Phillips, Lawrence, et al.
Pubblicazione: (2025)

Query, Don't Train: Privacy-Preserving Tabular Prediction from EHR Data via SQL Queries
di: Stoisser, Josefa Lia, et al.
Pubblicazione: (2025)

Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineering
di: Vijayvargiya, Sanidhya, et al.
Pubblicazione: (2025)

DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queries
di: Nam, Jaehyun, et al.
Pubblicazione: (2025)

WebDS: An End-to-End Benchmark for Web-based Data Science
di: Hsu, Ethan, et al.
Pubblicazione: (2025)

AmbigNLG: Addressing Task Ambiguity in Instruction for NLG
di: Niwa, Ayana, et al.
Pubblicazione: (2024)

Benchmarking Data Science Agents
di: Zhang, Yuge, et al.
Pubblicazione: (2024)

Ambig-IaC: Multi-level Disambiguation for Interactive Cloud Infrastructure-as-Code Synthesis
di: Yang, Zhenning, et al.
Pubblicazione: (2026)

DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning
di: Xie, Runpeng, et al.
Pubblicazione: (2025)

DataSciBench: An LLM Agent Benchmark for Data Science
di: Zhang, Dan, et al.
Pubblicazione: (2025)

Ambiguity Resolution with Human Feedback for Code Writing Tasks
di: Nandan, Aditey, et al.
Pubblicazione: (2025)

AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks
di: Jorf, Baraa Al, et al.
Pubblicazione: (2026)

TML-Bench: Benchmark for Data Science Agents on Tabular ML Tasks
di: Pinchuk, Mykola
Pubblicazione: (2026)

BioDSA-1K: Benchmarking Data Science Agents for Biomedical Research
di: Wang, Zifeng, et al.
Pubblicazione: (2025)

Human-Guided Agentic AI for Multimodal Clinical Prediction: Lessons from the AgentDS Healthcare Benchmark
di: Pulavarthy, Lalitha Pranathi, et al.
Pubblicazione: (2026)

To Be or Not To Be: Vector ontologies as a truly formal ontological framework
di: Rothenfusser, Kaspar
Pubblicazione: (2025)

TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks
di: Chu, Zhaoyang, et al.
Pubblicazione: (2026)

Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts
di: Wang, Boxuan, et al.
Pubblicazione: (2026)

DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
di: Huang, Yiming, et al.
Pubblicazione: (2024)

Framing AI System Benchmarking as a Learning Task: FlexBench and the Open MLPerf Dataset
di: Fursin, Grigori, et al.
Pubblicazione: (2025)

ART: Action-based Reasoning Task Benchmarking for Medical AI Agents
di: Mantravadi, Ananya, et al.
Pubblicazione: (2026)

MedSR-Impact: Transformer-Based Super-Resolution for Lung CT Segmentation, Radiomics, Classification, and Prognosis
di: Martell, Marc Boubnovski, et al.
Pubblicazione: (2025)

NEWSAGENT: Benchmarking Multimodal Agents as Journalists with Real-World Newswriting Tasks
di: Chien, Yen-Che, et al.
Pubblicazione: (2025)

LLM4DS: Evaluating Large Language Models for Data Science Code Generation
di: Nascimento, Nathalia, et al.
Pubblicazione: (2024)

A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents
di: Huang, Yuting, et al.
Pubblicazione: (2025)

AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
di: Lupidi, Alisia, et al.
Pubblicazione: (2026)

LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents
di: Choi, Jae-Woo, et al.
Pubblicazione: (2024)

Data Interpreter: An LLM Agent For Data Science
di: Hong, Sirui, et al.
Pubblicazione: (2024)

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution
di: Zhu, Zilin, et al.
Pubblicazione: (2026)

TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents
di: Liu, Zhiqiang, et al.
Pubblicazione: (2026)

Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity
di: Liang, Kaiqu, et al.
Pubblicazione: (2024)

Agent Mentor: Framing Agent Knowledge through Semantic Trajectory Analysis
di: Ben-Gigi, Roi, et al.
Pubblicazione: (2026)

R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science
di: Yang, Xu, et al.
Pubblicazione: (2025)

DSGym: A Holistic Framework for Evaluating and Training Data Science Agents
di: Nie, Fan, et al.
Pubblicazione: (2026)