Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929747681869824 |
|---|---|
| author | Liu, Xuanqing Kong, Luyang Niu, Wei Khashei, Afshin Zeng, Belinda Johnson, Steve Jay, Jon Golac, Davor Pope, Matt |
| author_facet | Liu, Xuanqing Kong, Luyang Niu, Wei Khashei, Afshin Zeng, Belinda Johnson, Steve Jay, Jon Golac, Davor Pope, Matt |
| contents | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. However, analyzing live dialogues in real-time necessitates low-latency processing systems, making it impractical to deploy models with billions of parameters due to latency constraints. As a result, practitioners often prefer smaller models with millions of parameters, trained on high-quality, human-annotated datasets. Yet, curating such datasets is both time-consuming and costly. Consequently, there is a growing need to combine the scalability of LLM-generated labels with the precision of human annotations, enabling fine-tuned smaller models to achieve both higher speed and accuracy comparable to larger models. In this paper, we introduce a simple yet effective framework to address this challenge. Our approach is specifically designed for per-utterance classification problems, which encompass tasks such as intent detection, dialogue state tracking, and more. To mitigate the impact of labeling errors from LLMs -- the primary source of inaccuracies in student models -- we propose a noise-reduced preference learning loss. Experimental results demonstrate that our method significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over $2\%$), dialogue act classification (over $1.5\%$), etc. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05620 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings Liu, Xuanqing Kong, Luyang Niu, Wei Khashei, Afshin Zeng, Belinda Johnson, Steve Jay, Jon Golac, Davor Pope, Matt Computation and Language Artificial Intelligence Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. However, analyzing live dialogues in real-time necessitates low-latency processing systems, making it impractical to deploy models with billions of parameters due to latency constraints. As a result, practitioners often prefer smaller models with millions of parameters, trained on high-quality, human-annotated datasets. Yet, curating such datasets is both time-consuming and costly. Consequently, there is a growing need to combine the scalability of LLM-generated labels with the precision of human annotations, enabling fine-tuned smaller models to achieve both higher speed and accuracy comparable to larger models. In this paper, we introduce a simple yet effective framework to address this challenge. Our approach is specifically designed for per-utterance classification problems, which encompass tasks such as intent detection, dialogue state tracking, and more. To mitigate the impact of labeling errors from LLMs -- the primary source of inaccuracies in student models -- we propose a noise-reduced preference learning loss. Experimental results demonstrate that our method significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over $2\%$), dialogue act classification (over $1.5\%$), etc. |
| title | Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.05620 |