LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909032927723520 |
|---|---|
| author | Steigerwald, Philipp Stieler, Mara Burghardt, Jennifer Rudolph, Eric Albrecht, Jens |
| author_facet | Steigerwald, Philipp Stieler, Mara Burghardt, Jennifer Rudolph, Eric Albrecht, Jens |
| contents | We demonstrate LLARS (LLM Assisted Research System), an open-source platform that bridges the gap between domain experts and developers for building LLM-based systems. It integrates three tightly connected modules into an end-to-end pipeline: Collaborative Prompt Engineering for real-time co-authoring with version control and instant LLM testing, Batch Generation for configurable output production across user-selected prompts $\times$ models $\times$ data with cost control, and Hybrid Evaluation where human and LLM evaluators jointly assess outputs through diverse assessment methods, with live agreement metrics and provenance analysis to identify the best model-prompt combination for a given use case. New prompts and models are automatically available for batch generation and completed batches can be turned into evaluation scenarios with a single click. Interviews with six domain experts and three developers in online counselling confirmed that LLARS feels intuitive, saves considerable time by keeping everything in one place and makes interdisciplinary collaboration seamless. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_10593 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation Steigerwald, Philipp Stieler, Mara Burghardt, Jennifer Rudolph, Eric Albrecht, Jens Artificial Intelligence Computation and Language Human-Computer Interaction Software Engineering We demonstrate LLARS (LLM Assisted Research System), an open-source platform that bridges the gap between domain experts and developers for building LLM-based systems. It integrates three tightly connected modules into an end-to-end pipeline: Collaborative Prompt Engineering for real-time co-authoring with version control and instant LLM testing, Batch Generation for configurable output production across user-selected prompts $\times$ models $\times$ data with cost control, and Hybrid Evaluation where human and LLM evaluators jointly assess outputs through diverse assessment methods, with live agreement metrics and provenance analysis to identify the best model-prompt combination for a given use case. New prompts and models are automatically available for batch generation and completed batches can be turned into evaluation scenarios with a single click. Interviews with six domain experts and three developers in online counselling confirmed that LLARS feels intuitive, saves considerable time by keeping everything in one place and makes interdisciplinary collaboration seamless. |
| title | LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation |
| topic | Artificial Intelligence Computation and Language Human-Computer Interaction Software Engineering |
| url | https://arxiv.org/abs/2605.10593 |