Large Language Models for Automatic Milestone Detection in Group Discussions
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913392340500480 |
|---|---|
| author | Duan, Zhuoxu Yang, Zhengye Westby, Samuel Riedl, Christoph Welles, Brooke Foucault Radke, Richard J. |
| author_facet | Duan, Zhuoxu Yang, Zhengye Westby, Samuel Riedl, Christoph Welles, Brooke Foucault Radke, Richard J. |
| contents | Large language models like GPT have proven widely successful on natural language understanding tasks based on written text documents. In this paper, we investigate an LLM's performance on recordings of a group oral communication task in which utterances are often truncated or not well-formed. We propose a new group task experiment involving a puzzle with several milestones that can be achieved in any order. We investigate methods for processing transcripts to detect if, when, and by whom a milestone has been completed. We demonstrate that iteratively prompting GPT with transcription chunks outperforms semantic similarity search methods using text embeddings, and further discuss the quality and randomness of GPT responses under different context window sizes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10842 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Models for Automatic Milestone Detection in Group Discussions Duan, Zhuoxu Yang, Zhengye Westby, Samuel Riedl, Christoph Welles, Brooke Foucault Radke, Richard J. Computation and Language Artificial Intelligence Human-Computer Interaction Large language models like GPT have proven widely successful on natural language understanding tasks based on written text documents. In this paper, we investigate an LLM's performance on recordings of a group oral communication task in which utterances are often truncated or not well-formed. We propose a new group task experiment involving a puzzle with several milestones that can be achieved in any order. We investigate methods for processing transcripts to detect if, when, and by whom a milestone has been completed. We demonstrate that iteratively prompting GPT with transcription chunks outperforms semantic similarity search methods using text embeddings, and further discuss the quality and randomness of GPT responses under different context window sizes. |
| title | Large Language Models for Automatic Milestone Detection in Group Discussions |
| topic | Computation and Language Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2406.10842 |