IHEval: Evaluating Language Models on Following the Instruction Hierarchy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913760616120320 |
|---|---|
| author | Zhang, Zhihan Li, Shiyang Zhang, Zixuan Liu, Xin Jiang, Haoming Tang, Xianfeng Gao, Yifan Li, Zheng Wang, Haodong Tan, Zhaoxuan Li, Yichuan Yin, Qingyu Yin, Bing Jiang, Meng |
| author_facet | Zhang, Zhihan Li, Shiyang Zhang, Zixuan Liu, Xin Jiang, Haoming Tang, Xianfeng Gao, Yifan Li, Zheng Wang, Haodong Tan, Zhaoxuan Li, Yichuan Yin, Qingyu Yin, Bing Jiang, Meng |
| contents | The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08745 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | IHEval: Evaluating Language Models on Following the Instruction Hierarchy Zhang, Zhihan Li, Shiyang Zhang, Zixuan Liu, Xin Jiang, Haoming Tang, Xianfeng Gao, Yifan Li, Zheng Wang, Haodong Tan, Zhaoxuan Li, Yichuan Yin, Qingyu Yin, Bing Jiang, Meng Computation and Language The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs. |
| title | IHEval: Evaluating Language Models on Following the Instruction Hierarchy |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.08745 |