Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.12925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911210978410496 |
|---|---|
| author | Akpinar, Nil-Jana Lee, Chia-Jung Murdock, Vanessa Perona, Pietro |
| author_facet | Akpinar, Nil-Jana Lee, Chia-Jung Murdock, Vanessa Perona, Pietro |
| contents | Large Language Models (LLMs) should answer factual questions truthfully, grounded in objective knowledge, regardless of user context such as self-disclosed personal information, or system personalization. In this paper, we present the first systematic evaluation of LLM robustness to inquiry personas, i.e. user profiles that convey attributes like identity, expertise, or belief. While prior work has primarily focused on adversarial inputs or distractors for robustness testing, we evaluate plausible, human-centered inquiry persona cues that users disclose in real-world interactions. We find that such cues can meaningfully alter QA accuracy and trigger failure modes such as refusals, hallucinated limitations, and role confusion. These effects highlight how model sensitivity to user framing can compromise factual reliability, and position inquiry persona testing as an effective tool for robustness evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12925 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering Akpinar, Nil-Jana Lee, Chia-Jung Murdock, Vanessa Perona, Pietro Computation and Language Machine Learning Large Language Models (LLMs) should answer factual questions truthfully, grounded in objective knowledge, regardless of user context such as self-disclosed personal information, or system personalization. In this paper, we present the first systematic evaluation of LLM robustness to inquiry personas, i.e. user profiles that convey attributes like identity, expertise, or belief. While prior work has primarily focused on adversarial inputs or distractors for robustness testing, we evaluate plausible, human-centered inquiry persona cues that users disclose in real-world interactions. We find that such cues can meaningfully alter QA accuracy and trigger failure modes such as refusals, hallucinated limitations, and role confusion. These effects highlight how model sensitivity to user framing can compromise factual reliability, and position inquiry persona testing as an effective tool for robustness evaluation. |
| title | Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2510.12925 |