PolicyPad: Collaborative Prototyping of LLM Policies
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911469175570432 |
|---|---|
| author | Feng, K. J. Kevin Kuo, Tzu-Sheng Chen, Quan Ze Cheong, Inyoung Holstein, Kenneth Zhang, Amy X. |
| author_facet | Feng, K. J. Kevin Kuo, Tzu-Sheng Chen, Quan Ze Cheong, Inyoung Holstein, Kenneth Zhang, Amy X. |
| contents | As LLMs gain adoption in high-stakes domains like mental health, domain experts are increasingly consulted to provide input into policies governing their behavior. From an observation of 19 policymaking workshops with 9 experts over 15 weeks, we identified opportunities to better support rapid experimentation, feedback, and iteration for collaborative policy design processes. We present PolicyPad, an interactive system that facilitates the emerging practice of LLM policy prototyping by drawing from established UX prototyping practices, including heuristic evaluation and storyboarding. Using PolicyPad, policy designers can collaborate on drafting a policy in real time while independently testing policy-informed model behavior with usage scenarios. We evaluate PolicyPad through workshops with 8 groups of 22 domain experts in mental health and law, finding that PolicyPad enhanced collaborative dynamics during policy design, enabled tight feedback loops, and led to novel policy contributions. Overall, our work paves expert-informed paths for advancing AI alignment and safety. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_19680 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PolicyPad: Collaborative Prototyping of LLM Policies Feng, K. J. Kevin Kuo, Tzu-Sheng Chen, Quan Ze Cheong, Inyoung Holstein, Kenneth Zhang, Amy X. Human-Computer Interaction Artificial Intelligence As LLMs gain adoption in high-stakes domains like mental health, domain experts are increasingly consulted to provide input into policies governing their behavior. From an observation of 19 policymaking workshops with 9 experts over 15 weeks, we identified opportunities to better support rapid experimentation, feedback, and iteration for collaborative policy design processes. We present PolicyPad, an interactive system that facilitates the emerging practice of LLM policy prototyping by drawing from established UX prototyping practices, including heuristic evaluation and storyboarding. Using PolicyPad, policy designers can collaborate on drafting a policy in real time while independently testing policy-informed model behavior with usage scenarios. We evaluate PolicyPad through workshops with 8 groups of 22 domain experts in mental health and law, finding that PolicyPad enhanced collaborative dynamics during policy design, enabled tight feedback loops, and led to novel policy contributions. Overall, our work paves expert-informed paths for advancing AI alignment and safety. |
| title | PolicyPad: Collaborative Prototyping of LLM Policies |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2509.19680 |