ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908433161125888 |
|---|---|
| author | Wu, Jiangxu Wang, Cong Su, TianHuang Yang, Jun Lin, Haozhi Zhang, Chao Peng, Ming Shi, Kai Yang, SongPan Pan, BinQing Li, ZiXian Yang, Ni Yang, ZhenYu |
| author_facet | Wu, Jiangxu Wang, Cong Su, TianHuang Yang, Jun Lin, Haozhi Zhang, Chao Peng, Ming Shi, Kai Yang, SongPan Pan, BinQing Li, ZiXian Yang, Ni Yang, ZhenYu |
| contents | The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coherence in multi-turn dialogues. Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. To address this, we propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. The framework iteratively refines instructions by incorporating Reviewer feedback, enhancing dialogue diversity and difficulty. We construct a multi-turn dataset using the Alpaca dataset and fine-tune the LLaMA2-13B model. Evaluations on MT-Bench, MMLU-Pro, and Auto-Arena demonstrate significant improvements, achieving absolute gains of 2.9\% on MMLU-Pro and 2\% on MT-Bench compared to prior state-of-the-art models based on LLaMA2-13B. Ablation studies confirm the critical role of the Review stage and the use of multiple Reviewers in boosting instruction diversity and difficulty. Our work highlights the potential of review-driven, multi-agent frameworks for generating high-quality conversational data at scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11010 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models Wu, Jiangxu Wang, Cong Su, TianHuang Yang, Jun Lin, Haozhi Zhang, Chao Peng, Ming Shi, Kai Yang, SongPan Pan, BinQing Li, ZiXian Yang, Ni Yang, ZhenYu Computation and Language Artificial Intelligence The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coherence in multi-turn dialogues. Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. To address this, we propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. The framework iteratively refines instructions by incorporating Reviewer feedback, enhancing dialogue diversity and difficulty. We construct a multi-turn dataset using the Alpaca dataset and fine-tune the LLaMA2-13B model. Evaluations on MT-Bench, MMLU-Pro, and Auto-Arena demonstrate significant improvements, achieving absolute gains of 2.9\% on MMLU-Pro and 2\% on MT-Bench compared to prior state-of-the-art models based on LLaMA2-13B. Ablation studies confirm the critical role of the Review stage and the use of multiple Reviewers in boosting instruction diversity and difficulty. Our work highlights the potential of review-driven, multi-agent frameworks for generating high-quality conversational data at scale. |
| title | ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.11010 |