Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.07642 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911199740821504 |
|---|---|
| author | Klisura, Đorđe Khoury, Joseph Kundu, Ashish Krishnan, Ram Rios, Anthony |
| author_facet | Klisura, Đorđe Khoury, Joseph Kundu, Ashish Krishnan, Ram Rios, Anthony |
| contents | Access control is a cornerstone of secure computing, yet large language models often blur role boundaries by producing unrestricted responses. We study role-conditioned refusals, focusing on the LLM's ability to adhere to access control policies by answering when authorized and refusing when not. To evaluate this behavior, we created a novel dataset that extends the Spider and BIRD text-to-SQL datasets, both of which have been modified with realistic PostgreSQL role-based policies at the table and column levels. We compare three designs: (i) zero or few-shot prompting, (ii) a two-step generator-verifier pipeline that checks SQL against policy, and (iii) LoRA fine-tuned models that learn permission awareness directly. Across multiple model families, explicit verification (the two-step framework) improves refusal precision and lowers false permits. At the same time, fine-tuning achieves a stronger balance between safety and utility (i.e., when considering execution accuracy). Longer and more complex policies consistently reduce the reliability of all systems. We release RBAC-augmented datasets and code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07642 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models Klisura, Đorđe Khoury, Joseph Kundu, Ashish Krishnan, Ram Rios, Anthony Computation and Language Access control is a cornerstone of secure computing, yet large language models often blur role boundaries by producing unrestricted responses. We study role-conditioned refusals, focusing on the LLM's ability to adhere to access control policies by answering when authorized and refusing when not. To evaluate this behavior, we created a novel dataset that extends the Spider and BIRD text-to-SQL datasets, both of which have been modified with realistic PostgreSQL role-based policies at the table and column levels. We compare three designs: (i) zero or few-shot prompting, (ii) a two-step generator-verifier pipeline that checks SQL against policy, and (iii) LoRA fine-tuned models that learn permission awareness directly. Across multiple model families, explicit verification (the two-step framework) improves refusal precision and lowers false permits. At the same time, fine-tuning achieves a stronger balance between safety and utility (i.e., when considering execution accuracy). Longer and more complex policies consistently reduce the reliability of all systems. We release RBAC-augmented datasets and code. |
| title | Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.07642 |