Can Hallucination Correction Improve Video-Language Alignment?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917931829428224 |
|---|---|
| author | Zhao, Lingjun Xie, Mingyang Cascante-Bonilla, Paola Daumé III, Hal Lee, Kwonjoon |
| author_facet | Zhao, Lingjun Xie, Mingyang Cascante-Bonilla, Paola Daumé III, Hal Lee, Kwonjoon |
| contents | Large Vision-Language Models often generate hallucinated content that is not grounded in its visual inputs. While prior work focuses on mitigating hallucinations, we instead explore leveraging hallucination correction as a training objective to improve video-language alignment. We introduce HACA, a self-training framework learning to correct hallucinations in descriptions that do not align with the video content. By identifying and correcting inconsistencies, HACA enhances the model's ability to align video and textual representations for spatio-temporal reasoning. Our experimental results show consistent gains in video-caption binding and text-to-video retrieval tasks, demonstrating that hallucination correction-inspired tasks serve as an effective strategy for improving vision and language alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15079 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can Hallucination Correction Improve Video-Language Alignment? Zhao, Lingjun Xie, Mingyang Cascante-Bonilla, Paola Daumé III, Hal Lee, Kwonjoon Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Large Vision-Language Models often generate hallucinated content that is not grounded in its visual inputs. While prior work focuses on mitigating hallucinations, we instead explore leveraging hallucination correction as a training objective to improve video-language alignment. We introduce HACA, a self-training framework learning to correct hallucinations in descriptions that do not align with the video content. By identifying and correcting inconsistencies, HACA enhances the model's ability to align video and textual representations for spatio-temporal reasoning. Our experimental results show consistent gains in video-caption binding and text-to-video retrieval tasks, demonstrating that hallucination correction-inspired tasks serve as an effective strategy for improving vision and language alignment. |
| title | Can Hallucination Correction Improve Video-Language Alignment? |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2502.15079 |