Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908978657624064 |
|---|---|
| author | Karnatak, Nimisha Chatila, Mohamad Hernández, Daniel Alejandro Pinzón Yazdanfar, Reza Dugas, Michelle Vakis, Renos |
| author_facet | Karnatak, Nimisha Chatila, Mohamad Hernández, Daniel Alejandro Pinzón Yazdanfar, Reza Dugas, Michelle Vakis, Renos |
| contents | General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA's multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4-3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized "evidence engine"; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for "ecosystem-aware" Humble AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_17843 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research Karnatak, Nimisha Chatila, Mohamad Hernández, Daniel Alejandro Pinzón Yazdanfar, Reza Dugas, Michelle Vakis, Renos Human-Computer Interaction Artificial Intelligence H.5.2; I.2.11 General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA's multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4-3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized "evidence engine"; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for "ecosystem-aware" Humble AI. |
| title | Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research |
| topic | Human-Computer Interaction Artificial Intelligence H.5.2; I.2.11 |
| url | https://arxiv.org/abs/2604.17843 |