RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914330246643712 |
|---|---|
| author | Man, Andrei Vlad Smădu, Răzvan-Alexandru Craciun, Cristian-George Cercel, Dumitru-Clementin Pop, Florin Cercel, Mihaela-Claudia |
| author_facet | Man, Andrei Vlad Smădu, Răzvan-Alexandru Craciun, Cristian-George Cercel, Dumitru-Clementin Pop, Florin Cercel, Mihaela-Claudia |
| contents | The intersection of AI and legal systems presents a growing need for tools that support legal education, particularly in under-resourced languages such as Romanian. In this work, we aim to evaluate the capabilities of Large Language Models (LLMs) and Vision-Language Models (VLMs) in understanding and reasoning about the Romanian driving law through textual and visual question-answering tasks. To facilitate this, we introduce RoD-TAL, a novel multimodal dataset comprising Romanian driving test questions, text-based and image-based, along with annotated legal references and explanations written by human experts. We implement and assess retrieval-augmented generation (RAG) pipelines, dense retrievers, and reasoning-optimized models across tasks, including Information Retrieval (IR), Question Answering (QA), Visual IR, and Visual QA. Our experiments demonstrate that domain-specific fine-tuning significantly enhances retrieval performance. At the same time, chain-of-thought prompting and specialized reasoning models improve QA accuracy, surpassing the minimum passing grades required for driving exams. We highlight the potential and limitations of applying LLMs and VLMs to legal education. We release the code and resources through the GitHub repository. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19666 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams Man, Andrei Vlad Smădu, Răzvan-Alexandru Craciun, Cristian-George Cercel, Dumitru-Clementin Pop, Florin Cercel, Mihaela-Claudia Computation and Language The intersection of AI and legal systems presents a growing need for tools that support legal education, particularly in under-resourced languages such as Romanian. In this work, we aim to evaluate the capabilities of Large Language Models (LLMs) and Vision-Language Models (VLMs) in understanding and reasoning about the Romanian driving law through textual and visual question-answering tasks. To facilitate this, we introduce RoD-TAL, a novel multimodal dataset comprising Romanian driving test questions, text-based and image-based, along with annotated legal references and explanations written by human experts. We implement and assess retrieval-augmented generation (RAG) pipelines, dense retrievers, and reasoning-optimized models across tasks, including Information Retrieval (IR), Question Answering (QA), Visual IR, and Visual QA. Our experiments demonstrate that domain-specific fine-tuning significantly enhances retrieval performance. At the same time, chain-of-thought prompting and specialized reasoning models improve QA accuracy, surpassing the minimum passing grades required for driving exams. We highlight the potential and limitations of applying LLMs and VLMs to legal education. We release the code and resources through the GitHub repository. |
| title | RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.19666 |