FLAIR: Feedback Learning for Adaptive Information Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909742745518080 |
|---|---|
| author | Zhang, William Zhu, Yiwen Lu, Yunlei Demarne, Mathieu Wang, Wenjing Deng, Kai Sahoo, Nutan Lin, Katherine Cilimdzic, Miso Krishnan, Subru |
| author_facet | Zhang, William Zhu, Yiwen Lu, Yunlei Demarne, Mathieu Wang, Wenjing Deng, Kai Sahoo, Nutan Lin, Katherine Cilimdzic, Miso Krishnan, Subru |
| contents | Recent advances in Large Language Models (LLMs) have driven the adoption of copilots in complex technical scenarios, underscoring the growing need for specialized information retrieval solutions. In this paper, we introduce FLAIR, a lightweight, feedback learning framework that adapts copilot systems' retrieval strategies by integrating domain-specific expert feedback. FLAIR operates in two stages: an offline phase obtains indicators from (1) user feedback and (2) questions synthesized from documentation, storing these indicators in a decentralized manner. An online phase then employs a two-track ranking mechanism to combine raw similarity scores with the collected indicators. This iterative setup refines retrieval performance for any query. Extensive real-world evaluations of FLAIR demonstrate significant performance gains on both previously seen and unseen queries, surpassing state-of-the-art approaches. The system has been successfully integrated into Copilot DECO, serving thousands of users at Microsoft, demonstrating its scalability and effectiveness in operational environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_13390 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FLAIR: Feedback Learning for Adaptive Information Retrieval Zhang, William Zhu, Yiwen Lu, Yunlei Demarne, Mathieu Wang, Wenjing Deng, Kai Sahoo, Nutan Lin, Katherine Cilimdzic, Miso Krishnan, Subru Information Retrieval H.3 Recent advances in Large Language Models (LLMs) have driven the adoption of copilots in complex technical scenarios, underscoring the growing need for specialized information retrieval solutions. In this paper, we introduce FLAIR, a lightweight, feedback learning framework that adapts copilot systems' retrieval strategies by integrating domain-specific expert feedback. FLAIR operates in two stages: an offline phase obtains indicators from (1) user feedback and (2) questions synthesized from documentation, storing these indicators in a decentralized manner. An online phase then employs a two-track ranking mechanism to combine raw similarity scores with the collected indicators. This iterative setup refines retrieval performance for any query. Extensive real-world evaluations of FLAIR demonstrate significant performance gains on both previously seen and unseen queries, surpassing state-of-the-art approaches. The system has been successfully integrated into Copilot DECO, serving thousands of users at Microsoft, demonstrating its scalability and effectiveness in operational environments. |
| title | FLAIR: Feedback Learning for Adaptive Information Retrieval |
| topic | Information Retrieval H.3 |
| url | https://arxiv.org/abs/2508.13390 |