Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.10507 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911288064475136 |
|---|---|
| author | He, Yun Li, Wenzhe Zhang, Hejia Li, Songlin Mandyam, Karishma Khosla, Sopan Xiong, Yuanhao Wang, Nanshu Peng, Xiaoliang Li, Beibin Bi, Shengjie Patil, Shishir G. Qi, Qi Feng, Shengyu Katz-Samuels, Julian Pang, Richard Yuanzhe Gonugondla, Sujan Lang, Hunter Yu, Yue Qian, Yundi Fazel-Zarandi, Maryam Yu, Licheng Benhalloum, Amine Awadalla, Hany Faruqui, Manaal |
| author_facet | He, Yun Li, Wenzhe Zhang, Hejia Li, Songlin Mandyam, Karishma Khosla, Sopan Xiong, Yuanhao Wang, Nanshu Peng, Xiaoliang Li, Beibin Bi, Shengjie Patil, Shishir G. Qi, Qi Feng, Shengyu Katz-Samuels, Julian Pang, Richard Yuanzhe Gonugondla, Sujan Lang, Hunter Yu, Yue Qian, Yundi Fazel-Zarandi, Maryam Yu, Licheng Benhalloum, Amine Awadalla, Hany Faruqui, Manaal |
| contents | Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10507 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following He, Yun Li, Wenzhe Zhang, Hejia Li, Songlin Mandyam, Karishma Khosla, Sopan Xiong, Yuanhao Wang, Nanshu Peng, Xiaoliang Li, Beibin Bi, Shengjie Patil, Shishir G. Qi, Qi Feng, Shengyu Katz-Samuels, Julian Pang, Richard Yuanzhe Gonugondla, Sujan Lang, Hunter Yu, Yue Qian, Yundi Fazel-Zarandi, Maryam Yu, Licheng Benhalloum, Amine Awadalla, Hany Faruqui, Manaal Computation and Language Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems. |
| title | AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.10507 |