Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911161390202880 |
|---|---|
| author | He, Bolei He, Xinran Shao, Run Shu, Shanfu Xue, Xianwei Cheng, Mingquan Li, Haifeng Ling, Zhenhua |
| author_facet | He, Bolei He, Xinran Shao, Run Shu, Shanfu Xue, Xianwei Cheng, Mingquan Li, Haifeng Ling, Zhenhua |
| contents | Large Language Models (LLMs) perform well in general QA but often struggle in domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces external knowledge but suffers from hallucinations and latency due to noisy retrievals. Continued pretraining internalizes domain knowledge but is costly and lacks cross-domain flexibility. We attribute this challenge to the long-tail distribution of domain knowledge, which leaves partial yet useful internal knowledge underutilized. We further argue that knowledge acquisition should be progressive, mirroring human learning: first understanding concepts, then applying them to complex reasoning. To address this, we propose Selct2Know (S2K), a cost-effective framework that internalizes domain knowledge through an internal-external knowledge self-selection strategy and selective supervised fine-tuning. We also introduce a structured reasoning data generation pipeline and integrate GRPO to enhance reasoning ability. Experiments on medical, legal, and financial QA benchmarks show that S2K consistently outperforms existing methods and matches domain-pretrained LLMs with significantly lower cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_15213 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering He, Bolei He, Xinran Shao, Run Shu, Shanfu Xue, Xianwei Cheng, Mingquan Li, Haifeng Ling, Zhenhua Computation and Language Large Language Models (LLMs) perform well in general QA but often struggle in domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces external knowledge but suffers from hallucinations and latency due to noisy retrievals. Continued pretraining internalizes domain knowledge but is costly and lacks cross-domain flexibility. We attribute this challenge to the long-tail distribution of domain knowledge, which leaves partial yet useful internal knowledge underutilized. We further argue that knowledge acquisition should be progressive, mirroring human learning: first understanding concepts, then applying them to complex reasoning. To address this, we propose Selct2Know (S2K), a cost-effective framework that internalizes domain knowledge through an internal-external knowledge self-selection strategy and selective supervised fine-tuning. We also introduce a structured reasoning data generation pipeline and integrate GRPO to enhance reasoning ability. Experiments on medical, legal, and financial QA benchmarks show that S2K consistently outperforms existing methods and matches domain-pretrained LLMs with significantly lower cost. |
| title | Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.15213 |