Context-Driven Interactive Query Simulations Based on Generative Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917575150010368 |
|---|---|
| author | Engelmann, Björn Breuer, Timo Friese, Jana Isabelle Schaer, Philipp Fuhr, Norbert |
| author_facet | Engelmann, Björn Breuer, Timo Friese, Jana Isabelle Schaer, Philipp Fuhr, Norbert |
| contents | Simulating user interactions enables a more user-oriented evaluation of information retrieval (IR) systems. While user simulations are cost-efficient and reproducible, many approaches often lack fidelity regarding real user behavior. Most notably, current user models neglect the user's context, which is the primary driver of perceived relevance and the interactions with the search results. To this end, this work introduces the simulation of context-driven query reformulations. The proposed query generation methods build upon recent Large Language Model (LLM) approaches and consider the user's context throughout the simulation of a search session. Compared to simple context-free query generation approaches, these methods show better effectiveness and allow the simulation of more efficient IR sessions. Similarly, our evaluations consider more interaction context than current session-based measures and reveal interesting complementary insights in addition to the established evaluation protocols. We conclude with directions for future work and provide an entirely open experimental setup. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_09631 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Context-Driven Interactive Query Simulations Based on Generative Large Language Models Engelmann, Björn Breuer, Timo Friese, Jana Isabelle Schaer, Philipp Fuhr, Norbert Information Retrieval Simulating user interactions enables a more user-oriented evaluation of information retrieval (IR) systems. While user simulations are cost-efficient and reproducible, many approaches often lack fidelity regarding real user behavior. Most notably, current user models neglect the user's context, which is the primary driver of perceived relevance and the interactions with the search results. To this end, this work introduces the simulation of context-driven query reformulations. The proposed query generation methods build upon recent Large Language Model (LLM) approaches and consider the user's context throughout the simulation of a search session. Compared to simple context-free query generation approaches, these methods show better effectiveness and allow the simulation of more efficient IR sessions. Similarly, our evaluations consider more interaction context than current session-based measures and reveal interesting complementary insights in addition to the established evaluation protocols. We conclude with directions for future work and provide an entirely open experimental setup. |
| title | Context-Driven Interactive Query Simulations Based on Generative Large Language Models |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2312.09631 |