Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.01081 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929331293388800 |
|---|---|
| author | Sudalairaj, Shivchander Bhandwaldar, Abhishek Pareja, Aldo Xu, Kai Cox, David D. Srivastava, Akash |
| author_facet | Sudalairaj, Shivchander Bhandwaldar, Abhishek Pareja, Aldo Xu, Kai Cox, David D. Srivastava, Akash |
| contents | This work introduces LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training. Leveraging a taxonomy-guided synthetic data generation process and a multi-phase tuning framework, LAB significantly reduces reliance on expensive human annotations and proprietary models like GPT-4. We demonstrate that LAB-trained models can achieve competitive performance across several benchmarks compared to models trained with traditional human-annotated or GPT-4 generated synthetic data. Thus offering a scalable, cost-effective solution for enhancing LLM capabilities and instruction-following behaviors without the drawbacks of catastrophic forgetting, marking a step forward in the efficient training of LLMs for a wide range of applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01081 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LAB: Large-Scale Alignment for ChatBots Sudalairaj, Shivchander Bhandwaldar, Abhishek Pareja, Aldo Xu, Kai Cox, David D. Srivastava, Akash Computation and Language Machine Learning This work introduces LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training. Leveraging a taxonomy-guided synthetic data generation process and a multi-phase tuning framework, LAB significantly reduces reliance on expensive human annotations and proprietary models like GPT-4. We demonstrate that LAB-trained models can achieve competitive performance across several benchmarks compared to models trained with traditional human-annotated or GPT-4 generated synthetic data. Thus offering a scalable, cost-effective solution for enhancing LLM capabilities and instruction-following behaviors without the drawbacks of catastrophic forgetting, marking a step forward in the efficient training of LLMs for a wide range of applications. |
| title | LAB: Large-Scale Alignment for ChatBots |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.01081 |