PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929388630573056 |
|---|---|
| author | Liu, An Yang, Zonghan Zhang, Zhenhe Hu, Qingyuan Li, Peng Yan, Ming Zhang, Ji Huang, Fei Liu, Yang |
| author_facet | Liu, An Yang, Zonghan Zhang, Zhenhe Hu, Qingyuan Li, Peng Yan, Ming Zhang, Ji Huang, Fei Liu, Yang |
| contents | While Large language models (LLMs) have demonstrated considerable capabilities across various natural language tasks, they often fall short of the performance achieved by domain-specific state-of-the-art models. One potential approach to enhance domain-specific capabilities of LLMs involves fine-tuning them using corresponding datasets. However, this method can be both resource and time-intensive, and not applicable to closed-source commercial LLMs. In this paper, we propose Preference Adaptation for Enhancing Domain-specific Abilities of LLMs (PANDA), a method designed to augment the domain-specific capabilities of LLMs by leveraging insights from the response preference of expert models without requiring fine-tuning. Our experimental results reveal that PANDA significantly enhances the domain-specific ability of LLMs on text classification and interactive decision tasks. Moreover, LLM with PANDA even outperforms the expert model that being learned on 4 tasks of ScienceWorld. This finding highlights the potential of exploring tuning-free approaches to achieve weak-to-strong generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_12835 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs Liu, An Yang, Zonghan Zhang, Zhenhe Hu, Qingyuan Li, Peng Yan, Ming Zhang, Ji Huang, Fei Liu, Yang Computation and Language Artificial Intelligence While Large language models (LLMs) have demonstrated considerable capabilities across various natural language tasks, they often fall short of the performance achieved by domain-specific state-of-the-art models. One potential approach to enhance domain-specific capabilities of LLMs involves fine-tuning them using corresponding datasets. However, this method can be both resource and time-intensive, and not applicable to closed-source commercial LLMs. In this paper, we propose Preference Adaptation for Enhancing Domain-specific Abilities of LLMs (PANDA), a method designed to augment the domain-specific capabilities of LLMs by leveraging insights from the response preference of expert models without requiring fine-tuning. Our experimental results reveal that PANDA significantly enhances the domain-specific ability of LLMs on text classification and interactive decision tasks. Moreover, LLM with PANDA even outperforms the expert model that being learned on 4 tasks of ScienceWorld. This finding highlights the potential of exploring tuning-free approaches to achieve weak-to-strong generalization. |
| title | PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.12835 |