generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914710995075072 |
|---|---|
| author | Spinner, Thilo Kehlbeck, Rebecca Sevastjanova, Rita Stähle, Tobias Keim, Daniel A. Deussen, Oliver El-Assady, Mennatallah |
| author_facet | Spinner, Thilo Kehlbeck, Rebecca Sevastjanova, Rita Stähle, Tobias Keim, Daniel A. Deussen, Oliver El-Assady, Mennatallah |
| contents | Large language models (LLMs) are widely deployed in various downstream tasks, e.g., auto-completion, aided writing, or chat-based text generation. However, the considered output candidates of the underlying search algorithm are under-explored and under-explained. We tackle this shortcoming by proposing a tree-in-the-loop approach, where a visual representation of the beam search tree is the central component for analyzing, explaining, and adapting the generated outputs. To support these tasks, we present generAItor, a visual analytics technique, augmenting the central beam search tree with various task-specific widgets, providing targeted visualizations and interaction possibilities. Our approach allows interactions on multiple levels and offers an iterative pipeline that encompasses generating, exploring, and comparing output candidates, as well as fine-tuning the model based on adapted data. Our case study shows that our tool generates new insights in gender bias analysis beyond state-of-the-art template-based methods. Additionally, we demonstrate the applicability of our approach in a qualitative user study. Finally, we quantitatively evaluate the adaptability of the model to few samples, as occurring in text-generation use cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07627 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation Spinner, Thilo Kehlbeck, Rebecca Sevastjanova, Rita Stähle, Tobias Keim, Daniel A. Deussen, Oliver El-Assady, Mennatallah Human-Computer Interaction Machine Learning I.2.7; H.5.2 Large language models (LLMs) are widely deployed in various downstream tasks, e.g., auto-completion, aided writing, or chat-based text generation. However, the considered output candidates of the underlying search algorithm are under-explored and under-explained. We tackle this shortcoming by proposing a tree-in-the-loop approach, where a visual representation of the beam search tree is the central component for analyzing, explaining, and adapting the generated outputs. To support these tasks, we present generAItor, a visual analytics technique, augmenting the central beam search tree with various task-specific widgets, providing targeted visualizations and interaction possibilities. Our approach allows interactions on multiple levels and offers an iterative pipeline that encompasses generating, exploring, and comparing output candidates, as well as fine-tuning the model based on adapted data. Our case study shows that our tool generates new insights in gender bias analysis beyond state-of-the-art template-based methods. Additionally, we demonstrate the applicability of our approach in a qualitative user study. Finally, we quantitatively evaluate the adaptability of the model to few samples, as occurring in text-generation use cases. |
| title | generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation |
| topic | Human-Computer Interaction Machine Learning I.2.7; H.5.2 |
| url | https://arxiv.org/abs/2403.07627 |