Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915926293610496 |
|---|---|
| author | Huang, Yue Zhuang, Haomin Ye, Jiayi Bao, Han Wang, Yanbo Hua, Hang Wu, Siyuan Chen, Pin-Yu Zhang, Xiangliang |
| author_facet | Huang, Yue Zhuang, Haomin Ye, Jiayi Bao, Han Wang, Yanbo Hua, Hang Wu, Siyuan Chen, Pin-Yu Zhang, Xiangliang |
| contents | Hard-gated safety checkers often over-refuse and misalign with a vendor's model spec; prevailing taxonomies also neglect robustness and honesty, yielding safer-on-paper yet less useful systems. This work introduces Guardian-as-an-Advisor (GaaA), a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference, keeping the base model operating under its original spec. To support training and evaluation, GuardSet is constructed, a 208k+ multi-domain dataset unifying harmful and harmless cases with targeted robustness and honesty slices. GuardAdvisor is trained via SFT followed by RL to enforce label-explanation consistency. GuardAdvisor attains competitive detection accuracy while enabling the advisory workflow; when used to augment inputs, responses improve over unaugmented prompts. A latency study shows advisor inference uses below 5% of base-model compute and adds only 2-10% end-to-end overhead under realistic harmful-input rates. Overall, GaaA steers models to comply with the model spec, maintaining safety while reducing over-refusal. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_07655 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs Huang, Yue Zhuang, Haomin Ye, Jiayi Bao, Han Wang, Yanbo Hua, Hang Wu, Siyuan Chen, Pin-Yu Zhang, Xiangliang Machine Learning Computation and Language Hard-gated safety checkers often over-refuse and misalign with a vendor's model spec; prevailing taxonomies also neglect robustness and honesty, yielding safer-on-paper yet less useful systems. This work introduces Guardian-as-an-Advisor (GaaA), a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference, keeping the base model operating under its original spec. To support training and evaluation, GuardSet is constructed, a 208k+ multi-domain dataset unifying harmful and harmless cases with targeted robustness and honesty slices. GuardAdvisor is trained via SFT followed by RL to enforce label-explanation consistency. GuardAdvisor attains competitive detection accuracy while enabling the advisory workflow; when used to augment inputs, responses improve over unaugmented prompts. A latency study shows advisor inference uses below 5% of base-model compute and adds only 2-10% end-to-end overhead under realistic harmful-input rates. Overall, GaaA steers models to comply with the model spec, maintaining safety while reducing over-refusal. |
| title | Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2604.07655 |