Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917689008586752 |
|---|---|
| author | Zhang, Yichi Chen, Zhuo Fang, Yin Lu, Yanxi Li, Fangming Zhang, Wen Chen, Huajun |
| author_facet | Zhang, Yichi Chen, Zhuo Fang, Yin Lu, Yanxi Li, Fangming Zhang, Wen Chen, Huajun |
| contents | Deploying large language models (LLMs) to real scenarios for domain-specific question answering (QA) is a key thrust for LLM applications, which poses numerous challenges, especially in ensuring that responses are both accommodating to user requirements and appropriately leveraging domain-specific knowledge bases. They are the two major difficulties for LLM application as vanilla fine-tuning falls short of addressing. Combining these requirements, we conceive of them as the requirement for the model's preference to be harmoniously aligned with humans'. Thus, we introduce Knowledgeable Preference AlignmenT (KnowPAT), which constructs two kinds of preference sets to tackle the two issues. Besides, we design a new alignment objective to align the LLM preference with different human preferences uniformly, aiming to optimize LLM performance in real-world, domain-specific QA settings. Adequate experiments and comprehensive comparisons with 15 baseline methods illustrate that our KnowPAT is a superior pipeline for real-scenario domain-specific QA with LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_06503 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering Zhang, Yichi Chen, Zhuo Fang, Yin Lu, Yanxi Li, Fangming Zhang, Wen Chen, Huajun Computation and Language Artificial Intelligence Deploying large language models (LLMs) to real scenarios for domain-specific question answering (QA) is a key thrust for LLM applications, which poses numerous challenges, especially in ensuring that responses are both accommodating to user requirements and appropriately leveraging domain-specific knowledge bases. They are the two major difficulties for LLM application as vanilla fine-tuning falls short of addressing. Combining these requirements, we conceive of them as the requirement for the model's preference to be harmoniously aligned with humans'. Thus, we introduce Knowledgeable Preference AlignmenT (KnowPAT), which constructs two kinds of preference sets to tackle the two issues. Besides, we design a new alignment objective to align the LLM preference with different human preferences uniformly, aiming to optimize LLM performance in real-world, domain-specific QA settings. Adequate experiments and comprehensive comparisons with 15 baseline methods illustrate that our KnowPAT is a superior pipeline for real-scenario domain-specific QA with LLMs. |
| title | Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2311.06503 |