Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.05497 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909491773046784 |
|---|---|
| author | Li, Yang Luo, Mingxuan Gong, Yeyun Lin, Chen Jiao, Jian Liu, Yi Huang, Kaili |
| author_facet | Li, Yang Luo, Mingxuan Gong, Yeyun Lin, Chen Jiao, Jian Liu, Yi Huang, Kaili |
| contents | Supervised fine-tuning with synthesized instructions has been a common practice for adapting LLMs to domain-specific QA tasks. However, the synthesized instructions deviate from real user questions and expected answers. This study proposes a novel framework called DeepThink to generate high-quality instructions. DeepThink first generates a few seed questions to mimic actual user questions, simulates conversations to uncover the hidden user needs, and refines the answer by conversational contexts and the retrieved documents for more comprehensive answers. Experiments demonstrate that DeepThink achieves an average performance improvement of 7.92% compared to a GPT-4-turbo+RAG-based assistant on the real user test set in the advertising domain across dimensions such as relevance, completeness, clarity, accuracy, and actionability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05497 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DeepThink: Aligning Language Models with Domain-Specific User Intents Li, Yang Luo, Mingxuan Gong, Yeyun Lin, Chen Jiao, Jian Liu, Yi Huang, Kaili Computation and Language Supervised fine-tuning with synthesized instructions has been a common practice for adapting LLMs to domain-specific QA tasks. However, the synthesized instructions deviate from real user questions and expected answers. This study proposes a novel framework called DeepThink to generate high-quality instructions. DeepThink first generates a few seed questions to mimic actual user questions, simulates conversations to uncover the hidden user needs, and refines the answer by conversational contexts and the retrieved documents for more comprehensive answers. Experiments demonstrate that DeepThink achieves an average performance improvement of 7.92% compared to a GPT-4-turbo+RAG-based assistant on the real user test set in the advertising domain across dimensions such as relevance, completeness, clarity, accuracy, and actionability. |
| title | DeepThink: Aligning Language Models with Domain-Specific User Intents |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.05497 |