Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917242092912640 |
|---|---|
| author | Hasan, Md. Toufique Khan, Ayman Asad Saari, Mika Bankhele, Vaishnavi Abrahamsson, Pekka |
| author_facet | Hasan, Md. Toufique Khan, Ayman Asad Saari, Mika Bankhele, Vaishnavi Abrahamsson, Pekka |
| contents | Large language models show promise for knowledge-intensive domains, yet their use in agriculture is constrained by weak grounding, English-centric training data, and limited real-world evaluation. These issues are amplified for low-resource languages, where high-quality domain documentation exists but remains difficult to access through general-purpose models. This paper presents AgriHubi, a domain-adapted retrieval-augmented generation (RAG) system for Finnish-language agricultural decision support. AgriHubi integrates Finnish agricultural documents with open PORO family models and combines explicit source grounding with user feedback to support iterative refinement. Developed over eight iterations and evaluated through two user studies, the system shows clear gains in answer completeness, linguistic accuracy, and perceived reliability. The results also reveal practical trade-offs between response quality and latency when deploying larger models. This study provides empirical guidance for designing and evaluating domain-specific RAG systems in low-resource language settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02208 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study Hasan, Md. Toufique Khan, Ayman Asad Saari, Mika Bankhele, Vaishnavi Abrahamsson, Pekka Computation and Language Artificial Intelligence Information Retrieval Software Engineering Large language models show promise for knowledge-intensive domains, yet their use in agriculture is constrained by weak grounding, English-centric training data, and limited real-world evaluation. These issues are amplified for low-resource languages, where high-quality domain documentation exists but remains difficult to access through general-purpose models. This paper presents AgriHubi, a domain-adapted retrieval-augmented generation (RAG) system for Finnish-language agricultural decision support. AgriHubi integrates Finnish agricultural documents with open PORO family models and combines explicit source grounding with user feedback to support iterative refinement. Developed over eight iterations and evaluated through two user studies, the system shows clear gains in answer completeness, linguistic accuracy, and perceived reliability. The results also reveal practical trade-offs between response quality and latency when deploying larger models. This study provides empirical guidance for designing and evaluating domain-specific RAG systems in low-resource language settings. |
| title | Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study |
| topic | Computation and Language Artificial Intelligence Information Retrieval Software Engineering |
| url | https://arxiv.org/abs/2602.02208 |