Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910878470766592 |
|---|---|
| author | Elizabeth, Michelle Veyret, Morgan Couceiro, Miguel Dusek, Ondrej Rojas-Barahona, Lina M. |
| author_facet | Elizabeth, Michelle Veyret, Morgan Couceiro, Miguel Dusek, Ondrej Rojas-Barahona, Lina M. |
| contents | Large language models (LLMs) gained immense popularity due to their impressive capabilities in unstructured conversations. Empowering LLMs with advanced prompting strategies such as reasoning and acting (ReAct) (Yao et al., 2022) has shown promise in solving complex tasks traditionally requiring reinforcement learning. In this work, we apply the ReAct strategy to guide LLMs performing task-oriented dialogue (TOD). We evaluate ReAct-based LLMs (ReAct-LLMs) both in simulation and with real users. While ReAct-LLMs severely underperform state-of-the-art approaches on success rate in simulation, this difference becomes less pronounced in human evaluation. Moreover, compared to the baseline, humans report higher subjective satisfaction with ReAct-LLM despite its lower success rate, most likely thanks to its natural and confidently phrased responses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01262 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings Elizabeth, Michelle Veyret, Morgan Couceiro, Miguel Dusek, Ondrej Rojas-Barahona, Lina M. Computation and Language Artificial Intelligence Human-Computer Interaction Large language models (LLMs) gained immense popularity due to their impressive capabilities in unstructured conversations. Empowering LLMs with advanced prompting strategies such as reasoning and acting (ReAct) (Yao et al., 2022) has shown promise in solving complex tasks traditionally requiring reinforcement learning. In this work, we apply the ReAct strategy to guide LLMs performing task-oriented dialogue (TOD). We evaluate ReAct-based LLMs (ReAct-LLMs) both in simulation and with real users. While ReAct-LLMs severely underperform state-of-the-art approaches on success rate in simulation, this difference becomes less pronounced in human evaluation. Moreover, compared to the baseline, humans report higher subjective satisfaction with ReAct-LLM despite its lower success rate, most likely thanks to its natural and confidently phrased responses. |
| title | Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings |
| topic | Computation and Language Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2412.01262 |