Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916974645215232 |
|---|---|
| author | Barbu, Eduard Dumitran, Adrian Marius |
| author_facet | Barbu, Eduard Dumitran, Adrian Marius |
| contents | Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23715 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering Barbu, Eduard Dumitran, Adrian Marius Computation and Language Machine Learning Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages. |
| title | Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2509.23715 |