ARGUS: Hallucination and Omission Evaluation in Video-LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909644952174592 |
|---|---|
| author | Rawal, Ruchit Shirkavand, Reza Huang, Heng Somepalli, Gowthami Goldstein, Tom |
| author_facet | Rawal, Ruchit Shirkavand, Reza Huang, Heng Somepalli, Gowthami Goldstein, Tom |
| contents | Video large language models have not yet been widely deployed, largely due to their tendency to hallucinate. Typical benchmarks for Video-LLMs rely simply on multiple-choice questions. Unfortunately, VideoLLMs hallucinate far more aggressively on freeform text generation tasks like video captioning than they do on multiple choice verification tasks. To address this weakness, we propose ARGUS, a VideoLLM benchmark that measures freeform video captioning performance. By comparing VideoLLM outputs to human ground truth captions, ARGUS quantifies dual metrics. First, we measure the rate of hallucinations in the form of incorrect statements about video content or temporal relationships. Second, we measure the rate at which the model omits important descriptive details. Together, these dual metrics form a comprehensive view of video captioning performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07371 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ARGUS: Hallucination and Omission Evaluation in Video-LLMs Rawal, Ruchit Shirkavand, Reza Huang, Heng Somepalli, Gowthami Goldstein, Tom Computer Vision and Pattern Recognition Video large language models have not yet been widely deployed, largely due to their tendency to hallucinate. Typical benchmarks for Video-LLMs rely simply on multiple-choice questions. Unfortunately, VideoLLMs hallucinate far more aggressively on freeform text generation tasks like video captioning than they do on multiple choice verification tasks. To address this weakness, we propose ARGUS, a VideoLLM benchmark that measures freeform video captioning performance. By comparing VideoLLM outputs to human ground truth captions, ARGUS quantifies dual metrics. First, we measure the rate of hallucinations in the form of incorrect statements about video content or temporal relationships. Second, we measure the rate at which the model omits important descriptive details. Together, these dual metrics form a comprehensive view of video captioning performance. |
| title | ARGUS: Hallucination and Omission Evaluation in Video-LLMs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.07371 |