PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910583807279104 |
|---|---|
| author | Wang, Yiying Li, Xiaojing Wang, Binzhu Zhou, Yueyang Lin, Yingru Ji, Han Chen, Hong Zhang, Jinshi Yu, Fei Zhao, Zewei Jin, Song Gong, Renji Xu, Wanqing |
| author_facet | Wang, Yiying Li, Xiaojing Wang, Binzhu Zhou, Yueyang Lin, Yingru Ji, Han Chen, Hong Zhang, Jinshi Yu, Fei Zhao, Zewei Jin, Song Gong, Renji Xu, Wanqing |
| contents | In domain-specific applications, GPT-4, augmented with precise prompts or Retrieval-Augmented Generation (RAG), shows notable potential but faces the critical tri-lemma of performance, cost, and data privacy. High performance requires sophisticated processing techniques, yet managing multiple agents within a complex workflow often proves costly and challenging. To address this, we introduce the PEER (Plan, Execute, Express, Review) multi-agent framework. This systematizes domain-specific tasks by integrating precise question decomposition, advanced information retrieval, comprehensive summarization, and rigorous self-assessment. Given the concerns of cost and data privacy, enterprises are shifting from proprietary models like GPT-4 to custom models, striking a balance between cost, security, and performance. We developed industrial practices leveraging online data and user feedback for efficient model tuning. This study provides best practice guidelines for applying multi-agent systems in domain-specific problem-solving and implementing effective agent tuning strategies. Our empirical studies, particularly in the financial question-answering domain, demonstrate that our approach achieves 95.0% of GPT-4's performance, while effectively managing costs and ensuring data privacy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_06985 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods Wang, Yiying Li, Xiaojing Wang, Binzhu Zhou, Yueyang Lin, Yingru Ji, Han Chen, Hong Zhang, Jinshi Yu, Fei Zhao, Zewei Jin, Song Gong, Renji Xu, Wanqing Artificial Intelligence In domain-specific applications, GPT-4, augmented with precise prompts or Retrieval-Augmented Generation (RAG), shows notable potential but faces the critical tri-lemma of performance, cost, and data privacy. High performance requires sophisticated processing techniques, yet managing multiple agents within a complex workflow often proves costly and challenging. To address this, we introduce the PEER (Plan, Execute, Express, Review) multi-agent framework. This systematizes domain-specific tasks by integrating precise question decomposition, advanced information retrieval, comprehensive summarization, and rigorous self-assessment. Given the concerns of cost and data privacy, enterprises are shifting from proprietary models like GPT-4 to custom models, striking a balance between cost, security, and performance. We developed industrial practices leveraging online data and user feedback for efficient model tuning. This study provides best practice guidelines for applying multi-agent systems in domain-specific problem-solving and implementing effective agent tuning strategies. Our empirical studies, particularly in the financial question-answering domain, demonstrate that our approach achieves 95.0% of GPT-4's performance, while effectively managing costs and ensuring data privacy. |
| title | PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2407.06985 |