How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911586416852992 |
|---|---|
| author | Xu, Ziwen Xu, Kewei Xu, Haoming Hong, Haiwen Huang, Longtao Xue, Hui Zhang, Ningyu Shen, Yongliang Zheng, Guozhou Chen, Huajun Deng, Shumin |
| author_facet | Xu, Ziwen Xu, Kewei Xu, Haoming Hong, Haiwen Huang, Longtao Xue, Hui Zhang, Ningyu Shen, Yongliang Zheng, Guozhou Chen, Huajun Deng, Shumin |
| contents | Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerEval, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02578 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities Xu, Ziwen Xu, Kewei Xu, Haoming Hong, Haiwen Huang, Longtao Xue, Hui Zhang, Ningyu Shen, Yongliang Zheng, Guozhou Chen, Huajun Deng, Shumin Computation and Language Artificial Intelligence Human-Computer Interaction Machine Learning Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerEval, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research. |
| title | How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities |
| topic | Computation and Language Artificial Intelligence Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2603.02578 |