ChatGPT for Conversational Recommendation: Refining Recommendations by Reprompting with Feedback
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914633835610112 |
|---|---|
| author | Spurlock, Kyle Dylan Acun, Cagla Saka, Esin Nasraoui, Olfa |
| author_facet | Spurlock, Kyle Dylan Acun, Cagla Saka, Esin Nasraoui, Olfa |
| contents | Recommendation algorithms have been pivotal in handling the overwhelming volume of online content. However, these algorithms seldom consider direct user input, resulting in superficial interaction between them. Efforts have been made to include the user directly in the recommendation process through conversation, but these systems too have had limited interactivity. Recently, Large Language Models (LLMs) like ChatGPT have gained popularity due to their ease of use and their ability to adapt dynamically to various tasks while responding to feedback. In this paper, we investigate the effectiveness of ChatGPT as a top-n conversational recommendation system. We build a rigorous pipeline around ChatGPT to simulate how a user might realistically probe the model for recommendations: by first instructing and then reprompting with feedback to refine a set of recommendations. We further explore the effect of popularity bias in ChatGPT's recommendations, and compare its performance to baseline models. We find that reprompting ChatGPT with feedback is an effective strategy to improve recommendation relevancy, and that popularity bias can be mitigated through prompt engineering. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_03605 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ChatGPT for Conversational Recommendation: Refining Recommendations by Reprompting with Feedback Spurlock, Kyle Dylan Acun, Cagla Saka, Esin Nasraoui, Olfa Information Retrieval Artificial Intelligence Computation and Language Machine Learning I.2.7; H.3.3 Recommendation algorithms have been pivotal in handling the overwhelming volume of online content. However, these algorithms seldom consider direct user input, resulting in superficial interaction between them. Efforts have been made to include the user directly in the recommendation process through conversation, but these systems too have had limited interactivity. Recently, Large Language Models (LLMs) like ChatGPT have gained popularity due to their ease of use and their ability to adapt dynamically to various tasks while responding to feedback. In this paper, we investigate the effectiveness of ChatGPT as a top-n conversational recommendation system. We build a rigorous pipeline around ChatGPT to simulate how a user might realistically probe the model for recommendations: by first instructing and then reprompting with feedback to refine a set of recommendations. We further explore the effect of popularity bias in ChatGPT's recommendations, and compare its performance to baseline models. We find that reprompting ChatGPT with feedback is an effective strategy to improve recommendation relevancy, and that popularity bias can be mitigated through prompt engineering. |
| title | ChatGPT for Conversational Recommendation: Refining Recommendations by Reprompting with Feedback |
| topic | Information Retrieval Artificial Intelligence Computation and Language Machine Learning I.2.7; H.3.3 |
| url | https://arxiv.org/abs/2401.03605 |