BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914693066522624 |
|---|---|
| author | Ren, Ruiyang Qiu, Peng Qu, Yingqi Liu, Jing Zhao, Wayne Xin Wu, Hua Wen, Ji-Rong Wang, Haifeng |
| author_facet | Ren, Ruiyang Qiu, Peng Qu, Yingqi Liu, Jing Zhao, Wayne Xin Wu, Hua Wen, Ji-Rong Wang, Haifeng |
| contents | Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation. Considering the scarcity and limit (e.g., privacy issues) of real user data, in this paper, we conduct large-scale user simulation for web search, to improve the analysis and modeling of user search behavior. Specially, we propose BASES, a novel user simulation framework with LLM-based agents, designed to facilitate comprehensive simulations of web search user behaviors. Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. To demonstrate the effectiveness of BASES, we conduct evaluation experiments based on two human benchmarks in both Chinese and English, demonstrating that BASES can effectively simulate large-scale human-like search behaviors. To further accommodate the research on web search, we develop WARRIORS, a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions, which can greatly bolster research in the field of information retrieval. Our code and data will be publicly released soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17505 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BASES: Large-scale Web Search User Simulation with Large Language Model based Agents Ren, Ruiyang Qiu, Peng Qu, Yingqi Liu, Jing Zhao, Wayne Xin Wu, Hua Wen, Ji-Rong Wang, Haifeng Information Retrieval Computation and Language Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation. Considering the scarcity and limit (e.g., privacy issues) of real user data, in this paper, we conduct large-scale user simulation for web search, to improve the analysis and modeling of user search behavior. Specially, we propose BASES, a novel user simulation framework with LLM-based agents, designed to facilitate comprehensive simulations of web search user behaviors. Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. To demonstrate the effectiveness of BASES, we conduct evaluation experiments based on two human benchmarks in both Chinese and English, demonstrating that BASES can effectively simulate large-scale human-like search behaviors. To further accommodate the research on web search, we develop WARRIORS, a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions, which can greatly bolster research in the field of information retrieval. Our code and data will be publicly released soon. |
| title | BASES: Large-scale Web Search User Simulation with Large Language Model based Agents |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2402.17505 |