AgentInstruct: Toward Generative Teaching with Agentic Flows
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929410213412864 |
|---|---|
| author | Mitra, Arindam Del Corro, Luciano Zheng, Guoqing Mahajan, Shweti Rouhana, Dany Codas, Andres Lu, Yadong Chen, Wei-ge Vrousgos, Olga Rosset, Corby Silva, Fillipe Khanpour, Hamed Lara, Yash Awadallah, Ahmed |
| author_facet | Mitra, Arindam Del Corro, Luciano Zheng, Guoqing Mahajan, Shweti Rouhana, Dany Codas, Andres Lu, Yadong Chen, Wei-ge Vrousgos, Olga Rosset, Corby Silva, Fillipe Khanpour, Hamed Lara, Yash Awadallah, Ahmed |
| contents | Synthetic data is becoming increasingly important for accelerating the development of language models, both large and small. Despite several successful use cases, researchers also raised concerns around model collapse and drawbacks of imitating other models. This discrepancy can be attributed to the fact that synthetic data varies in quality and diversity. Effective use of synthetic data usually requires significant human effort in curating the data. We focus on using synthetic data for post-training, specifically creating data by powerful models to teach a new skill or behavior to another model, we refer to this setting as Generative Teaching. We introduce AgentInstruct, an extensible agentic framework for automatically creating large amounts of diverse and high-quality synthetic data. AgentInstruct can create both the prompts and responses, using only raw data sources like text documents and code files as seeds. We demonstrate the utility of AgentInstruct by creating a post training dataset of 25M pairs to teach language models different skills, such as text editing, creative writing, tool usage, coding, reading comprehension, etc. The dataset can be used for instruction tuning of any base model. We post-train Mistral-7b with the data. When comparing the resulting model Orca-3 to Mistral-7b-Instruct (which uses the same base model), we observe significant improvements across many benchmarks. For example, 40% improvement on AGIEval, 19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and 45% improvement on AlpacaEval. Additionally, it consistently outperforms other models such as LLAMA-8B-instruct and GPT-3.5-turbo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03502 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AgentInstruct: Toward Generative Teaching with Agentic Flows Mitra, Arindam Del Corro, Luciano Zheng, Guoqing Mahajan, Shweti Rouhana, Dany Codas, Andres Lu, Yadong Chen, Wei-ge Vrousgos, Olga Rosset, Corby Silva, Fillipe Khanpour, Hamed Lara, Yash Awadallah, Ahmed Artificial Intelligence Computation and Language Machine Learning Synthetic data is becoming increasingly important for accelerating the development of language models, both large and small. Despite several successful use cases, researchers also raised concerns around model collapse and drawbacks of imitating other models. This discrepancy can be attributed to the fact that synthetic data varies in quality and diversity. Effective use of synthetic data usually requires significant human effort in curating the data. We focus on using synthetic data for post-training, specifically creating data by powerful models to teach a new skill or behavior to another model, we refer to this setting as Generative Teaching. We introduce AgentInstruct, an extensible agentic framework for automatically creating large amounts of diverse and high-quality synthetic data. AgentInstruct can create both the prompts and responses, using only raw data sources like text documents and code files as seeds. We demonstrate the utility of AgentInstruct by creating a post training dataset of 25M pairs to teach language models different skills, such as text editing, creative writing, tool usage, coding, reading comprehension, etc. The dataset can be used for instruction tuning of any base model. We post-train Mistral-7b with the data. When comparing the resulting model Orca-3 to Mistral-7b-Instruct (which uses the same base model), we observe significant improvements across many benchmarks. For example, 40% improvement on AGIEval, 19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and 45% improvement on AlpacaEval. Additionally, it consistently outperforms other models such as LLAMA-8B-instruct and GPT-3.5-turbo. |
| title | AgentInstruct: Toward Generative Teaching with Agentic Flows |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.03502 |