From RAG to Agentic RAG for Faithful Islamic Question Answering
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917196127535104 |
|---|---|
| author | Bhatia, Gagan Mubarak, Hamdy Jarrar, Mustafa Mikros, George Zaraket, Fadi Alhirthani, Mahmoud Al-Khatib, Mutaz Cochrane, Logan Darwish, Kareem Yahiaoui, Rashid Alam, Firoj |
| author_facet | Bhatia, Gagan Mubarak, Hamdy Jarrar, Mustafa Mikros, George Zaraket, Fadi Alhirthani, Mahmoud Al-Khatib, Mutaz Cochrane, Logan Darwish, Kareem Yahiaoui, Rashid Alam, Firoj |
| contents | LLMs are increasingly used for Islamic question answering, where ungrounded responses may carry serious religious consequences. Yet standard MCQ/MRC-style evaluations do not capture key real-world failure modes, notably free-form hallucinations and whether models appropriately abstain when evidence is lacking. To shed a light on this aspect we introduce ISLAMICFAITHQA, a 3,810-item bilingual (Arabic/English) generative benchmark with atomic single-gold answers, which enables direct measurement of hallucination and abstention. We additionally developed an end-to-end grounded Islamic modelling suite consisting of (i) 25K Arabic text-grounded SFT reasoning pairs, (ii) 5K bilingual preference samples for reward-guided alignment, and (iii) a verse-level Qur'an retrieval corpus of $\sim$6k atomic verses (ayat). Building on these resources, we develop an agentic Quran-grounding framework (agentic RAG) that uses structured tool calls for iterative evidence seeking and answer revision. Experiments across Arabic-centric and multilingual LLMs show that retrieval improves correctness and that agentic RAG yields the largest gains beyond standard RAG, achieving state-of-the-art performance and stronger Arabic-English robustness even with a small model (i.e., Qwen3 4B). We will make the experimental resources and datasets publicly available for the community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_07528 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From RAG to Agentic RAG for Faithful Islamic Question Answering Bhatia, Gagan Mubarak, Hamdy Jarrar, Mustafa Mikros, George Zaraket, Fadi Alhirthani, Mahmoud Al-Khatib, Mutaz Cochrane, Logan Darwish, Kareem Yahiaoui, Rashid Alam, Firoj Computation and Language Artificial Intelligence LLMs are increasingly used for Islamic question answering, where ungrounded responses may carry serious religious consequences. Yet standard MCQ/MRC-style evaluations do not capture key real-world failure modes, notably free-form hallucinations and whether models appropriately abstain when evidence is lacking. To shed a light on this aspect we introduce ISLAMICFAITHQA, a 3,810-item bilingual (Arabic/English) generative benchmark with atomic single-gold answers, which enables direct measurement of hallucination and abstention. We additionally developed an end-to-end grounded Islamic modelling suite consisting of (i) 25K Arabic text-grounded SFT reasoning pairs, (ii) 5K bilingual preference samples for reward-guided alignment, and (iii) a verse-level Qur'an retrieval corpus of $\sim$6k atomic verses (ayat). Building on these resources, we develop an agentic Quran-grounding framework (agentic RAG) that uses structured tool calls for iterative evidence seeking and answer revision. Experiments across Arabic-centric and multilingual LLMs show that retrieval improves correctness and that agentic RAG yields the largest gains beyond standard RAG, achieving state-of-the-art performance and stronger Arabic-English robustness even with a small model (i.e., Qwen3 4B). We will make the experimental resources and datasets publicly available for the community. |
| title | From RAG to Agentic RAG for Faithful Islamic Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.07528 |