Image de couverture de livre

PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lu, Yunan, Liu, Luigi, Yahia, Omar, Sharma, Arpit, Yu, Zhou
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LocalRQA: From Generating Data to Locally Training, Testing, and Deploying Retrieval-Augmented QA Systems
par: Yu, Xiao, et autres
Publié: (2024)

SAGE: A Top-Down Bottom-Up Knowledge-Grounded User Simulator for Multi-turn AGent Evaluation
par: Shea, Ryan, et autres
Publié: (2025)

pdfQA: Diverse, Challenging, and Realistic Question Answering over PDFs
par: Schimanski, Tobias, et autres
Publié: (2026)

NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts
par: Gupta, Abhay, et autres
Publié: (2025)

Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks
par: Hegazy, Mahmood
Publié: (2024)

One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement
par: Zhou, Yixiao, et autres
Publié: (2026)

Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents
par: Chopra, Harshita, et autres
Publié: (2026)

The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution
par: Li, Junlong, et autres
Publié: (2025)

YIELD: A Large-Scale Dataset and Evaluation Framework for Information Elicitation Agents
par: De Lima, Victor, et autres
Publié: (2026)

Tip of the Tongue Query Elicitation for Simulated Evaluation
par: He, Yifan, et autres
Publié: (2025)

Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guarantees
par: Montreuil, Yannis, et autres
Publié: (2024)

FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes
par: Constantinides, Christodoulos, et autres
Publié: (2025)

Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA
par: Abbas, Ummar, et autres
Publié: (2026)

DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation
par: Khan, Saadat Hasan, et autres
Publié: (2026)

Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries
par: Liu, Gabrielle Kaili-May, et autres
Publié: (2025)

Diversity Measures: Domain-Independent Proxies for Failure in Language Model Queries
par: Ngu, Noel, et autres
Publié: (2023)

QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA
par: Dineen, Jacob, et autres
Publié: (2025)

Synthetic Users, Real Differences: an Evaluation Framework for User Simulation in Multi-Turn Conversations
par: Liu, Yu Lu, et autres
Publié: (2026)

Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intents
par: Wang, Shuting, et autres
Publié: (2025)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
par: Zhang, Xuemiao, et autres
Publié: (2025)

Contextual Multilingual Spellchecker for User Queries
par: Sharma, Sanat, et autres
Publié: (2023)

Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generation
par: Hu, Zhe, et autres
Publié: (2024)

REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
par: Liang, Buyun, et autres
Publié: (2026)

An Answer is just the Start: Related Insight Generation for Open-Ended Document-Grounded QA
par: Sharma, Saransh, et autres
Publié: (2026)

heiDS at ArchEHR-QA 2025: From Fixed-k to Query-dependent-k for Retrieval Augmented Generation
par: Chouhan, Ashish, et autres
Publié: (2025)

Elicitation Matters: How Prompts and Query Protocols Shape LLM Surrogates under Sparse Observations
par: Lei, Ge, et autres
Publié: (2026)

BioGraphletQA: Knowledge-Anchored Generation of Complex QA Datasets
par: Jonker, Richard A. A., et autres
Publié: (2026)

Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents
par: Jain, Akriti, et autres
Publié: (2026)

Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation
par: Ren, Zhiyao, et autres
Publié: (2024)

Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
par: Lee, Dongryeol, et autres
Publié: (2026)

Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA
par: Li, Wenwei, et autres
Publié: (2026)

EduVidQA: Generating and Evaluating Long-form Answers to Student Questions based on Lecture Videos
par: Ray, Sourjyadip, et autres
Publié: (2025)

Personalized LLM for Generating Customized Responses to the Same Query from Different Users
par: Zeng, Hang, et autres
Publié: (2024)

RJUA-QA: A Comprehensive QA Dataset for Urology
par: Lyu, Shiwei, et autres
Publié: (2023)

On the Performance of an Explainable Language Model on PubMedQA
par: Srinivasan, Venkat, et autres
Publié: (2025)

Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis
par: B, Srihari K, et autres
Publié: (2025)

Towards Better Question Generation in QA-based Event Extraction
par: Hong, Zijin, et autres
Publié: (2024)

From Query to Counsel: Structured Reasoning with a Multi-Agent Framework and Dataset for Legal Consultation
par: Lu, Mingfei, et autres
Publié: (2026)

SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent
par: Ye, Jing, et autres
Publié: (2024)

From Failure to Mastery: Generating Hard Samples for Tool-use Agents
par: Hao, Bingguang, et autres
Publié: (2026)